Parameter Extraction and Performance Analysis of 3D Surface Reconstruction Techniques

To evaluate whether a computer vision model can utilize monocular endoscopic images of the middle ear to generate three-dimensional (3D) surface reconstructions that register to ground-truth (GT) imaging with clinically acceptable accuracy for surgical navigation. Eight temporal bone phantoms derived from the publicly available OpenEar data set were 3D-printed. Middle ear endoscopy was performed in each model using a standard monocular otologic endoscope, and a single endoscopic image captured of the mesotympanum. Images were processed with a monocular surface reconstruction model to generate predicted 3D surface reconstructions. Reconstructions were registered to GT meshes using natural anatomic landmarks. Surface reconstruction error (SRE) was quantified using the mean modified Hausdorff distance, and target registration error (TRE) was calculated using withheld test fiducials. Mean SRE across models was 0.58 ± 0.10mm, with 81.6% of points within 1mm of GT surfaces. Mean TRE was 0.96 ± 0.17mm. Simple 2D, monocular endoscopic images can be converted into high-fidelity 3D surface reconstructions of middle ear anatomy, which can be registered to ground-truth imaging with submillimeter registration accuracy. These findings establish the preclinical feasibility of monocular, endoscope-centric navigation for transcanal endoscopic ear surgery.

Three-dimensional Feature Recognition-based Automated Patient Treatment Mismatch Verification System for Radiation Therapy

The existing surface reconstruction algorithms currently reconstruct large amounts of mesh data. Consequently, many of these algorithms cannot meet the efficiency requirements of real-time data transmission in a web environment. This paper proposes a lightweight surface reconstruction method for online 3D scanned point cloud data oriented toward 3D printing. The proposed online lightweight surface reconstruction algorithm is composed of a point cloud update algorithm (PCU), a rapid iterative closest point algorithm (RICP), and an improved Poisson surface reconstruction algorithm (IPSR). The generated lightweight point cloud data are pretreated using an updating and rapid registration method. The Poisson surface reconstruction is also accomplished by a pretreatment to recompute the point cloud normal vectors; this approach is based on a least squares method, and the postprocessing of the PDE patch generation was based on biharmonic-like fourth-order PDEs, which effectively reduces the amount of reconstructed mesh data and improves the efficiency of the algorithm. This method was verified using an online personalized customization system that was developed with WebGL and oriented toward 3D printing. The experimental results indicate that this method can generate a lightweight 3D scanning mesh rapidly and efficiently in a web environment.

This article presents a new stereo matching algorithm based on local method. The absolute difference (AD) algorithm works fast and precise at low texture area, however this algorithm is sensitive to radiometric distortions (i.e., contrast or brightness) and low texture areas. To overcome these problems, the proposed algorithm utilizes a combination of AD, gradient difference (GM) and census transform (CT). The GM algorithm is robust against the radiometric errors and CT algorithm works well on the low texture regions. The proposed algorithm performs much better based on the experimental results of the Middlebury dataset. The disparity map from the result consists of depth information which requires for the three-dimensional (3D) surface reconstruction.

Surface reconstruction is a foundational topic in computer graphics and has gained substantial research interest in recent years. With the emergence of advanced neural radiance fields (NeRFs) and 3D Gaussian splatting (3D GS), numerous innovative many novel algorithms for 3D model surface reconstruction have been developed. The rapid expansion of this field presents challenges in tracking ongoing advancements. This survey aims to present core methodologies for the surface reconstruction of 3D models and establish a structured roadmap that encompasses 3D representations, reconstruction methods, datasets, and related applications. Specifically, we introduce 3D representations using 3D Gaussians as the central framework. Additionally, we provide a comprehensive overview of the rapidly evolving surface reconstruction methods based on 3D Gaussian splatting. We categorize the primary phases of surface reconstruction algorithms for 3D models into scene representation, Gaussian optimization, and surface structure extraction. Finally, we review the available datasets, applications, and challenges and suggest potential future research directions in this domain. Through this survey, we aim to provide valuable resources that support and inspire researchers in the field, fostering advancements in 3D reconstruction technologies.

This paper presents a 3D soft tissue surface reconstruction method based on improved compressed sensing and radial basis function interpolation for a small amount of uniform sampling data points on 3D surface. We adopt radial basis function interpolation to obtain the same amount of data points as to be reconstructed and propose an improved compressed sensing method to reconstruct 3D surface: we design a deterministic measurement matrix to signal observation, and then adopt the discrete cosine transform to the 3D coordinate sparse representation and use weak choose regularized orthogonal matching pursuit algorithm to reconstruct. Experimental results show that the proposed algorithm improves the resolution of the surface as well as the accuracy. The average maximum error is less than 0.9012 mm, which is smooth enough to provide accurate surface data model for virtual reality based surgery system.KeywordsCompressed sensingDeterministic measurement matrixRadial basis function interpolationSparse 3D discrete point

A fast surface reconstruction method for fluorescence molecular tomography based on cross-beam edge back projection

Abstract. In this paper, we propose the 3D surface reconstruction scheme using multi-view and multi-date Google Earth (GE) satellite images. Multi-view stereo matching (MVS) scheme is one of the methods for reconstructing dense 3D surface based on multi-view images and corresponding camera pose geometry. If many views are input, MVS can estimate the disparity (depth) by matching pixels. However, the common users are not always possible to obtain both multi-view satellite images and the camera geometry (such as Rational Polynomial Camera) in various earth regions. Instead, the GE provides multi-view and multi-date satellite images of earth regions. Therefore, the goal of the proposed method is to perform a 3D surface reconstruction using the GE satellite image. We suppose that the GE satellite image is a pinhole camera model, and the camera pose geometry is estimated using the perspective projection model (PPM) based structure from the motion (SfM) method. Then the 3D surface is reconstructed and fusion using the MVS method. However, the GE satellite image is a transformed pseudo-orthoimage for integration into the raster image. For this reason, the camera pose geometry is inaccurately computed in the SfM process. Thus, the high-rise structures in the reconstructed 3D surface are distorted (distorted hexahedral 3D space). Importantly, the satellite image is a weak PPM and it can express the orthographic projection model. Therefore, we compute 3D homography for transforming between distorted hexahedral space to orthographic cuboid space. Then, the distorted 3D surface is transformed using a projective reconstruction based on 3D homography. The transformed 3D surface has the correct shape in the orthographic projection model. The advantage of the proposed method is that the 3D surface of various earth regions is reconstructed using simply accessible GE satellite images. And the transformed 3D surface is reconstructed into orthographic projection model space, thus the orthoimage can be generated using projection.

Structure-From-Motion for Earthwork Planning Khaled Nassar, Ebrahim A.Aly, Younghan Jung Pages 310-316 (2011 Proceedings of the 28th ISARC, Seoul, Korea, ISBN 978-89-954572-4-5, ISSN 2413-5844) Abstract: The reconstruction of surfaces from unorganized point clouds can be of great use in construction applications. Point clouds are mostly created by 3D scanners. Another technique for generating point clouds is the structure-from-motion technique, where the point cloud is generated from a sequence of images. This paper utilizes a surface reconstruction technique to model and quantify earthwork. The described technique can be used in preliminary planning, for project updates, estimating of earthwork quantities, as well as for embedded planning systems in construction equipment. The paper examines the use of structure-from-motion techniques in earthwork application and identifies limitations and, pros, and cons of this approach. Data from 23 earthwork excavation construction sites were collected and analyzed. 3D surface reconstructions were applied on the 23 sites during the construction phases and 3D surfaces were reconstructed and compared to the original land form. In addition, similar experiments were conducted with piles of earth. The results were analyzed to determine appropriate ranges of use for structure-from-motion surface reconstructions in earthwork applications. Results show that the technique is most suited for pit excavations with areas less than 2000 meter squared and with depths up to 5 meters. In addition, piles up to 10 meter highs and with base areas up to 300 meters were successfully reconstructed. The results of this paper should be of great importance to contractors willing to utilize this technology in everyday use. Keywords: Structure-from-motion, Construction, Earthwork, Image Processing, 3D Surface Reconstruction DOI: https://doi.org/10.22260/ISARC2011/0055 Download fulltext Download BibTex Download Endnote (RIS) TeX Import to Mendeley

Contact pattern measurement is commonly applied to assess gear transmission and meshing performance in aviation spiral bevel gear (ASBG) manufacturing enterprises. However, these patterns cannot be directly segmented and obtained from point clouds or images because each tooth of the spiral bevel gear has complex 3D spiral surface and texture information. Moreover, research on the high-precision 3D colour reconstruction of spiral gear tooth surfaces is relatively rare. Aimed at these problems, the current work proposed a high-precision 3D colour reconstruction method of spiral bevel gear. This method is based on the fusion of the image and point-cloud information. It includes two core steps to satisfy the practical demands of high precision and high efficiency. First, dense point clouds of the 3D calibration block are converted to a plane using the bird’s eye view method for corner detection. Second, the corners are quickly and accurately detected using a two-stage corner detection algorithm based on the inherent geometric features of the calibration block. The identified corners are then used to solve the transformation matrix from the pixel coordinate system to the line laser coordinate system. Image textures can be precisely projected and fused onto a 3D geometry for the coloured reconstruction of the gear tooth surface on the basis of this transformation matrix. The proposed method was validated by applying it to the 3D colour reconstruction of tooth surfaces of 10 distinct types of ASBGs. The experimental results demonstrated that the re-projection error of the proposed method is less than 0.1 millimetres. The minimal margin of error enables high-precision 3D colour reconstruction of the spiral bevel gear tooth surface. Compared to other classic 3D colour reconstruction methods, the proposed method achieves superior accuracy and efficiency. An experimental device was also developed for the 3D colour reconstruction of the tooth surface of ASBGs, as illustrated in figure 1.

Fire detection and 3D surface reconstruction based on stereoscopic pictures and probabilistic fuzzy logic

In this paper we present a novel image-based 3D surface reconstruction technique that incorporates both reflectance and polarisation features into a variational framework. The proposed technique is suitable for single-image and multi-image (photopolarimetric stereo) analysis. It is especially suited for the difficult task of 3D reconstruction of rough metallic surfaces. An error functional consisting of several error terms related to the measured reflectance and polarisation properties is minimised in order to obtain a 3D reconstruction of the surface. We show that the combined approach strongly increases the accuracy of the surface reconstruction result, compared to techniques based on either reflectance or polarisation alone. We perform an evaluation of the algorithm with respect to single and multiple reflectance and polarisation images of the surface, relying on synthetic ground truth data. This evaluation also reveals which polarisation features should preferably be used in the context of 3D reconstruction of rough metallic surfaces. Furthermore, we report 3D reconstruction results for a raw forged iron surface, thus showing the applicability of our method in real-world scenarios, here in the domain of industrial quality inspection.

Three-dimensional (3D) medical images are prone to overlap, and there are some problems, such as low detection efficiency and inconsistent with the actual situation. Therefore, a 3D medical image surface reconstruction method based on data mining and machine learning is proposed. The 3D medical images were classified according to different ways, the information frame of 3D medical images was established and the surface overlapping information model of 3D images was given. Based on this information framework, the nonlinear function of overlapping area information of 3D medical images was constructed. The weight of the nonlinear function was used to calculate the input and output results of overlapping area information. Combined with the input mode of 3D medical image information, the error between the information output and the expected output was set. The nonlinear function weight of the overlapping area information of 3D medical images was modified by using the learning rate and the use time of the overlapping area information, and the influence factors of the overlapping information detection were obtained by increasing the situation terms, so as to complete the detection of the surface reconstruction information of 3D medical images. The experimental results show that the information detection results of the proposed method fit well with the actual situation, and the information detection efficiency is high.

Recently it has been thought possible to characterise a pulmonary lesion by its vascular involvement. The complex vascular relationships are difficult to demonstrate by the "normal" technique of transverse CT sections. Spiral CT with 3D-surface reconstruction makes it possible to evaluate the vessels accurately and simulates the appearances one would have by looking into the thorax with a demonstration of all vessels lying within the pulmonary space examined by CT. Venous involvement was shown in every one of 18 malignant lesions but was also found in 4 out of 11 benign lesions, arterial involvement occurred in 14 out of 18 malignant and in 2 benign lesions. CT evidence of vascular involvement is in favour of a malignant tumour but is no absolute proof.

3DSEM++: Adaptive and intelligent 3D SEM surface reconstruction