Fluorescence molecular tomography with optimal radon transform based surface reconstruction

Accurate small animal surface reconstruction is important for full angle non-contact fluorescence molecular tomography (FMT) systems. In this letter, an optimal surface reconstruction method for FMT is proposed. The proposed method uses a line search method to minimize the mismatch between the reconstructed three-dimensional (3D) surface and the projected object silhouette at different angles. The results show that the mean mismatches of the 3D surfaces generated on three live anesthetized mice are all less than two charge coupled device (CCD) pixels (0.154 mm). With the accurately reconstructed 3D surface, in-vivo FMT is also performed.

A fast surface reconstruction method for fluorescence molecular tomography based on cross-beam edge back projection

Abstract. In this paper, we propose the 3D surface reconstruction scheme using multi-view and multi-date Google Earth (GE) satellite images. Multi-view stereo matching (MVS) scheme is one of the methods for reconstructing dense 3D surface based on multi-view images and corresponding camera pose geometry. If many views are input, MVS can estimate the disparity (depth) by matching pixels. However, the common users are not always possible to obtain both multi-view satellite images and the camera geometry (such as Rational Polynomial Camera) in various earth regions. Instead, the GE provides multi-view and multi-date satellite images of earth regions. Therefore, the goal of the proposed method is to perform a 3D surface reconstruction using the GE satellite image. We suppose that the GE satellite image is a pinhole camera model, and the camera pose geometry is estimated using the perspective projection model (PPM) based structure from the motion (SfM) method. Then the 3D surface is reconstructed and fusion using the MVS method. However, the GE satellite image is a transformed pseudo-orthoimage for integration into the raster image. For this reason, the camera pose geometry is inaccurately computed in the SfM process. Thus, the high-rise structures in the reconstructed 3D surface are distorted (distorted hexahedral 3D space). Importantly, the satellite image is a weak PPM and it can express the orthographic projection model. Therefore, we compute 3D homography for transforming between distorted hexahedral space to orthographic cuboid space. Then, the distorted 3D surface is transformed using a projective reconstruction based on 3D homography. The transformed 3D surface has the correct shape in the orthographic projection model. The advantage of the proposed method is that the 3D surface of various earth regions is reconstructed using simply accessible GE satellite images. And the transformed 3D surface is reconstructed into orthographic projection model space, thus the orthoimage can be generated using projection.

Fluorescence molecular imaging (FMI) has shown potential to detect and delineate cancer during surgery or diagnostic endoscopy. Recent progress on imaging systems has allowed sensitive detection of fluorescent agents even in video rate mode. However, lack of standardization in fluorescence imaging challenges the clinical application of FMI, since the use of different systems may lead to different results from a given study, even when using the same fluorescent agent. In this work, we investigate the use of a composite fluorescence phantom, employed as an FMI standard, to offer a comprehensive method for validation and standardization of the performance of different imaging systems. To exclude user interaction, all phantom features are automatically extracted from the acquired epi-illumination color and fluorescence images, using appropriately constructed templates. These features are then employed to characterize the performance and compare different cameras to each other. The proposed method could serve as a framework toward the calibration and benchmarking of FMI systems, to facilitate their clinical translation.

Fluorescence molecular tomography (FMT) could exploit the distribution of fluorescent biomarkers that target tumors accurately and effectively, which enables noninvasive real-time 3-D visualization as well as quantitative analysis of small tumors in small animal studies in vivo. Due to the difficulties of reconstruction, continuous efforts are being made to find more practical and efficient approaches to accurately obtain the characteristics of fluorescent regions inside biological tissues. In this paper, we propose a region reconstruction method for FMT, which is defined as an L1-norm regularization piecewise constant level set approach. The proposed approach adopts a priori information including the sparsity of the fluorescent sources and the fluorescent contrast between the target and background. When the contrast of different fluorescent sources is low to a certain degree, our approach can simultaneously solve the detection and characterization problems for the reconstruction of FMT. To evaluate the performance of the region reconstruction method, numerical phantom experiments and in vivo bead-implanted mouse experiments were performed. The results suggested that the proposed region reconstruction method was able to reconstruct the features of the fluorescent regions accurately and effectively, and the proposed method was able to be feasibly adopted in in vivo application.

BackgroundImplicit shape-based reconstruction method in fluorescence molecular tomography (FMT) is capable of achieving higher image clarity than image-based reconstruction method. However, the implicit shape method suffers from a low convergence speed and performs unstably due to the utilization of gradient-based optimization methods. Moreover, the implicit shape method requires priori information about the number of targets.MethodsA shape-based reconstruction scheme of FMT with a cosinoidal level set method is proposed in this paper. The Heaviside function in the classical implicit shape method is replaced with a cosine function, and then the reconstruction can be accomplished with the Levenberg–Marquardt method rather than gradient-based methods. As a result, the priori information about the number of targets is not required anymore and the choice of step length is avoided.ResultsNumerical simulations and phantom experiments were carried out to validate the proposed method. Results of the proposed method show higher contrast to noise ratios and Pearson correlations than the implicit shape method and image-based reconstruction method. Moreover, the number of iterations required in the proposed method is much less than the implicit shape method.ConclusionsThe proposed method performs more stably, provides a faster convergence speed than the implicit shape method, and achieves higher image clarity than the image-based reconstruction method.

This paper is about automatically reconstructing the full 3D surface of an object observed in motion by a single static camera. Based on the two paradigms, structure from motion and linear intensity subspaces, we introduce the geotensity constraint that governs the relationship between four or more images of a moving object. We show that it is possible in theory to solve for 3D Lambertian surface structure for the case of a single point light source and propose that a solution exists for an arbitrary number point light sources. The surface may or may not be textured. We then give an example of automatic surface reconstruction of a face under a point light source using arbitrary unknown object motion and a single fixed camera.

In this chapter, the algorithms to reconstruct the shape of the visible 3D surface using only the brightness variation information (shading, that is, the level variations of gray or colored) observed in the image are described. The inverse problem of reconstructing the shape of the visible surface from the changes in brightness in the image is known as the Shape from Shading (SfS) problem. The reconstruction of the visible surface should not be strictly understood as a 3D reconstruction of the surface. In fact, from a single point of the observation of a scene, a monocular vision system cannot estimate a distance measure between the observer and a visible object, so with the SfS algorithms there is a nonmetric but qualitative reconstruction of the 3D surface. The theory of the SfS is described, which is based on the knowledge of the light source (direction and distribution), the model of reflectance of the scene, the observation point, and the geometry of the visible surface, which together contributes to the image formation process. The relationships between the light intensity values of the image and the geometry of the visible surface are derived (in terms of the orientation of the surface, point by point) under some lighting conditions and the reflectance model. Other 3D surface reconstruction algorithms based on the Shape from XXX paradigm are also described, where XXX can be texture, structured light projected onto the surface to be reconstructed, or 2D images of the focused or defocused surface.

An experimental structured light projection system which includes a miniaturized projector is described. The system has been designed to be integrated in a fluorescence molecular tomography (FMT) prototype in order to reconstruct the surface of mice and phantom studies. A high-accuracy phase map is retrieved with phase-shifted sinusoidal fringes. Phase error due to the nonlinear gamma function of the pico-projector is calibrated and compensated. Robust phase unwrapping is performed with an additional Gray-code projection sequence. An automatic phase-to-height non-linear calibration scheme has been applied using objects located in the extremes of the field of view. The accuracy of the proposed method has been tested with a realistic mouse model and ray-tracing software.

The purpose of our project is to develop an image guided system for the medialization laryngoplasty. One of the fundamental challenges in our system is to accurately register the preoperative 3D CT data to the intraoperative 3D surfaces of the patient. In this paper, we will present a combined surface and fiducial based registration method to register the preoperative 3D CT data to the intraoperative surface of larynx. To accurately model the exposed surface area, an active illumination based stereo vision technique is used for the surface reconstruction. To register the point clouds from the intraoperative stage to the preoperative 3D CT data, a shape priori based ICP method is proposed to quickly register the two surfaces. The proposed approach is capable of tracking the fiducial markers and reconstructing the surface of larynx with no damage to the anatomical structure. Although, the proposed method is specifically designed for the image guided laryngoplasty, it can be applied to other image guided surgical areas. We used off-the-shelf digital cameras, LCD projector and rapid 3D prototyper to develop our experimental system. The final RMS error in the registration is less than 1mm.KeywordsImage Guided Surgery3D ReconstructionRegistration

Accurate and rapid alignment of laser scanned 3D surface using TSK-type neural-fuzzy network-based coarse-to-fine strategy

Methods of 3D reconstruction of medical images mainly included surface and volume reconstruction at present. In the experiment of 3D ultrasound reconstruction for spine of people, the result using volume reconstruction was unsatisfactory because of the ultrasound images' characteristic. So, we brought forward a method which was based on contour line to reconstruct 3D surface of spine. In this paper, we collected spinal ultrasound images using 3D freehand ultrasound firstly; next, tracing spinal contour line of each ultrasound images manually; finally, we used these contour lines to reconstruct spinal surface. The result showed that spinal surface reconstruction based on 2D contour line was close to real spinal surface, it could give physicians a better help in diagnosis of the spine.

This paper presents a 3D soft tissue surface reconstruction method based on improved compressed sensing and radial basis function interpolation for a small amount of uniform sampling data points on 3D surface. We adopt radial basis function interpolation to obtain the same amount of data points as to be reconstructed and propose an improved compressed sensing method to reconstruct 3D surface: we design a deterministic measurement matrix to signal observation, and then adopt the discrete cosine transform to the 3D coordinate sparse representation and use weak choose regularized orthogonal matching pursuit algorithm to reconstruct. Experimental results show that the proposed algorithm improves the resolution of the surface as well as the accuracy. The average maximum error is less than 0.9012 mm, which is smooth enough to provide accurate surface data model for virtual reality based surgery system.KeywordsCompressed sensingDeterministic measurement matrixRadial basis function interpolationSparse 3D discrete point

To evaluate whether a computer vision model can utilize monocular endoscopic images of the middle ear to generate three-dimensional (3D) surface reconstructions that register to ground-truth (GT) imaging with clinically acceptable accuracy for surgical navigation. Eight temporal bone phantoms derived from the publicly available OpenEar data set were 3D-printed. Middle ear endoscopy was performed in each model using a standard monocular otologic endoscope, and a single endoscopic image captured of the mesotympanum. Images were processed with a monocular surface reconstruction model to generate predicted 3D surface reconstructions. Reconstructions were registered to GT meshes using natural anatomic landmarks. Surface reconstruction error (SRE) was quantified using the mean modified Hausdorff distance, and target registration error (TRE) was calculated using withheld test fiducials. Mean SRE across models was 0.58 ± 0.10mm, with 81.6% of points within 1mm of GT surfaces. Mean TRE was 0.96 ± 0.17mm. Simple 2D, monocular endoscopic images can be converted into high-fidelity 3D surface reconstructions of middle ear anatomy, which can be registered to ground-truth imaging with submillimeter registration accuracy. These findings establish the preclinical feasibility of monocular, endoscope-centric navigation for transcanal endoscopic ear surgery.

Surface reconstruction is a foundational topic in computer graphics and has gained substantial research interest in recent years. With the emergence of advanced neural radiance fields (NeRFs) and 3D Gaussian splatting (3D GS), numerous innovative many novel algorithms for 3D model surface reconstruction have been developed. The rapid expansion of this field presents challenges in tracking ongoing advancements. This survey aims to present core methodologies for the surface reconstruction of 3D models and establish a structured roadmap that encompasses 3D representations, reconstruction methods, datasets, and related applications. Specifically, we introduce 3D representations using 3D Gaussians as the central framework. Additionally, we provide a comprehensive overview of the rapidly evolving surface reconstruction methods based on 3D Gaussian splatting. We categorize the primary phases of surface reconstruction algorithms for 3D models into scene representation, Gaussian optimization, and surface structure extraction. Finally, we review the available datasets, applications, and challenges and suggest potential future research directions in this domain. Through this survey, we aim to provide valuable resources that support and inspire researchers in the field, fostering advancements in 3D reconstruction technologies.