Fast-SNARF: A Fast Deformer for Articulated Neural Fields.

With the increasing use of image-based 3D reconstruction in medical procedures, accurate scene reconstruction plays a crucial role in surgical navigation and assisted treatment. However, the monotonous colors, limited image features, and obvious brightness fluctuations of thoracoscopic scenes make the feature point matching process, on which traditional 3D reconstruction methods rely, unstable and unreliable. It brings a great challenge to accurate 3D reconstruction. In this study, a new method for implicit 3D reconstruction of monocular thoracoscopic scenes is proposed. The method combines a pre-trained metric depth estimation model with neural radiation field (NeRF) technique and uses dense SLAM to accurately compute the camera pose. To ensure the accuracy of the depth values and the structural consistency of the reconstructed scene, depth and normal constraints are added to the original color constraints of the NeRF network to achieve high-quality scene reconstruction results. We conducted experiments on the SCARED dataset and the clinical dataset. After comparing with other methods, the depth estimation accuracy and point cloud reconstruction quality of this paper outperform the existing methods. The method in this paper can provide more accurate 3D reconstruction of complex thoracic surgical scenes, which can significantly improve the accuracy and therapeutic efficacy of surgical navigation.

Neural fields are a new (and old!) approach to solving problems over spacetime via first-order optimization of a neural network. Over the past three years, combining neural fields with classic computer graphics approaches have allowed us to make significant advances in solving computer vision problems like scene reconstruction. I will present recent work that can reconstruct indoor scenes for photorealistic interactive exploration using new scalable hybrid neural field representations. This has applications where any real-world place needs to be digitized, especially for visualization purposes.

In the field of dynamic human synthesis, some recent works try to decompose a non-rigidly deforming scene into a canonical neural radiance field and use a set of deformation fields for mapping observation-space points to the canonical space, thereby enabling them to learn the dynamic scene from images. Due to the highly under-constrained optimization cased by deformation field without prior and the insufficient of surface appearance information cased by sparse views, the rendering result exists obvious appearance artifacts. In this paper, to address the problem of artifacts, we present a novel method called UV-guided Neural Radiance Fields (UVNeRF), consisting of three modules: Canonical Space Mapping Module (CSMM), Texture Space Mapping Module (TSMM), UV-guided Neural Rendering Module (UVNRM). CSMM map observation-space points to the canonical space based 3D human skeletons which can regularize learning of the deformation field. TSMM map canonical-space points to the texture space for obtaining a rough human surface representation on the UV space as the extra information. UVNRM render the image result using the outputs of CSMM and TSMM. The experimental studies on Human3.6M and ZJU-MoCap dataset show that our approach gains noteworthy enhancements comparing recent dynamic human synthesis methods.

VR and AR not only expand the art field, but also enrich the form and content of digital media art and promote the development of digital media art. This paper proposes a 3D reconstruction of media art scenes based on the improved Neural Radiation Field (NeRF) technology and then completes the style migration, through which a unique scene art effect is formed, with a view to exploring new ways for the development of new digital media art forms. In the virtual orchard construction, the PSNR value of the optimized NeRF method is improved by 0.205, the SSIM value is improved by 0.015, and the LPIPS value is reduced by 0.018 compared with the state-of-the-art method, which can be concluded that the optimized NeRF method has a good image synthesis effect.

In this paper, we propose possibility for reconstruction of surface of an underwater object or 3D scene reconstruction of an underwater environment using an economical RGB-D sensor such as Microsoft Kinect. Reconstructing the 3D surface of an underwater object is a challenging task due to degraded quality of underwater images. There are various reasons of quality degradation of underwater images i.e., non-uniform illumination of light on the surface of objects, scattering and absorption effects. Particles and impurities present in underwater produces Gaussian noise on the captured underwater optical images which degrades the quality of images. However, using depth sensors, as a cost effective alternative, we aim to show that underwater 3D scene reconstruction is possible with sight tradeoffs on accuracy but major cost saving. The acquired depth data is proposed to be processed by applying real-time mesh generating techniques from the acquired point cloud. The experimental result aims to show that the proposed method reconstructs 3D surface of underwater objects accurately using captured underwater depth images.

With the rapid advancement of Unmanned Aerial Vehicle applications, vision-based 3D scene reconstruction has demonstrated significant value in fields such as remote sensing and target detection. However, scenes captured by UAVs are often large-scale, sparsely viewed, and complex. These characteristics pose significant challenges for neural radiance field (NeRF)-based reconstruction. Specifically, the reconstructed images may suffer from blurred edges and unclear textures. This is primarily due to the lack of edge information and the fact that certain objects appear in only a few images, leading to incomplete reconstructions. To address these issues, this paper proposes a hybrid image encoder that combines convolutional neural networks and Transformer to extract image features to assist NeRF in scene reconstruction and generate new perspective images. Furthermore, we extend the NeRF architecture by introducing an additional branch that estimates uncertainty values associated with transient regions in the scene, enabling the model to suppress dynamic content and focus on static structure reconstruction. To further improve synthesis quality, we also refine the loss function used during training to better guide network optimization. Experimental results on a custom UAV aerial imagery dataset demonstrate the effectiveness of our method in accurately reconstructing and rendering UAV-captured scenes.

Substantial progress has been made on modeling rigid 3D objects using deep implicit representations. Yet, extending these methods to learn neural models of human shape is still in its infancy. Human bodies are complex and the key challenge is to learn a representation that generalizes such that it can express body shape deformations for unseen subjects in unseen, highly-articulated, poses. To address this challenge, we introduce LEAP (LEarning Articulated occupancy of People), a novel neural occupancy representation of the human body. Given a set of bone transformations (i.e. joint locations and rotations) and a query point in space, LEAP first maps the query point to a canonical space via learned linear blend skinning (LBS) functions and then efficiently queries the occupancy value via an occupancy network that models accurate identity- and pose-dependent deformations in the canonical space. Experiments show that our canonicalized occupancy estimation with the learned LBS functions greatly improves the generalization capability of the learned occupancy representation across various human shapes and poses, outperforming existing solutions in all settings.

This paper systematically reviews common 3D model representation methods, including point clouds, voxels, meshes, boundary representations (B-Rep), and implicit representations such as signed distance fields (SDF) and neural radiance fields (NeRF), and introduces their differences in geometric accuracy, computational efficiency, and application scenarios. Furthermore, it explores conversion techniques between point clouds and meshes, point clouds and B-Rep, voxels and meshes, meshes and B-Rep, as well as meshes and implicit representations, covering both traditional geometric algorithms and deep learning-based intelligent approaches. The analysis evaluates the strengths and weaknesses of existing methods in terms of computational precision, topological consistency, and efficiency, highlighting key challenges such as low processing efficiency for high-resolution models and difficulties in preserving complex topological structures. The analysis indicates that future research should focus on developing multimodal fusion representation methods, optimizing high-precision conversion algorithms, developing dedicated conversion technologies for specific industries, and actively exploring AI-driven intelligent conversion paradigms to provide key technical support for the innovative development of fields such as intelligent manufacturing and digital twins.

The extraction of multi-view matching points is one of the key elements in 3D reconstruction of multi-view image scene, because the extraction results will directly affect the accuracy of 3D reconstruction. With the conversion from the extraction of multi-view matching points to dynamic connectivity, a solution based on the Union Find algorithm was designed. The efficient tree structure with parent-link was used to organize the nodes in the Union Find sets, so that it was only needed to modify the addressing parameter of a single node in each process of adding the matching points pair, which avoided the computational process of the traversal array to compare with addressing parameter and improved the efficiency to find and modify. At the same time, the weighted method was applied to optimize the algorithm and the weighted encoding method was used to replace the commonly used hard encoding, which can balance the dendrogram structure and reduce the average depth of nodes in the tree. Experimental results of multiple groups of image sets showed that, compared with the traditional breadth-first search algorithm, the algorithm based on Union Find had higher reliability and computational efficiency.

DWT-3DRec: DeepJSCC-based wireless transmission for efficient 3D scene reconstruction using CityNeRF

Over the last two decades, research community of computer vision has developed various techniques suitable for underwater applications using intensity images. This chapter will explore 3D reconstruction of underwater natural scenes and objects based on stereo vision, which will be helpful in mine detection, inspection of shipwrecks, detection of telecommunication cables and pipelines. The general steps involved in 3D reconstruction using stereo vision are provided. The brief summary of papers for 3D reconstruction of underwater environment is presented. 3D reconstruction of underwater natural scenes and objects is challenging problem due to light propagation in underwater. In contrast to light propagation in the air, the light rays are attenuated and scattered, having a great effect on image quality. We have proposed preprocessing technique to enhance degraded underwater images. At the end of the chapter, we have presented the proposed stereo vision based 3D reconstruction technique to reconstruct 3D surface of underwater objects. Ultimately, this chapter intends to give an overview of the 3D reconstruction technique using stereo vision in order to help a reader in understanding stereo vision and its benefits for underwater applications.

This Targeting to the complexity and large amount of 3D(three-dimensional) reconstruction of complex scene in virtual environment, Firstly, The 3D data acquisition and conversion principle, scene modeling and texture mapping method were described. Secondly, the geometric modeling and image fusion method was presented. Thirdly, the reconstruction mathematical model of image based on entropy variation were introduced, and then data compression algorithm to reduce the number of the entity grid was given. Besides, the intricate and fine animal wood carving model on the roof was an example, which was optimized and reconstructed. At last, a classical Chinese architecture was used as a prototype for the fusion modeling and optimization, and merging with virtual software a 3D reconstruction of classical architecture and the scene Navigation system in virtual environment was developed with VC++.

NeRF has garnered extensive attention from researchers due to its impressive performance in three-dimensional scene reconstruction and realistic rendering. It is perceived as a potential pivotal technology for scene reconstruction in fields such as virtual reality and augmented reality. However, most NeRF-related research and applications heavily rely on precise pose data. The challenge of effectively reconstructing scenes in situations with inaccurate or missing pose data remains pressing. To address this issue, we examine the relationship between different resolution encodings and pose estimation and introduce BiResNeRF, a scene reconstruction method based on both low and high-resolution hash encoding modules, accompanied by a two-stage training strategy. The training strategy includes setting different learning rates and sampling strategies for different stages, designing stage transition signals, and implementing a smooth warm-up learning rate scheduling strategy after the phase transition. The experimental results indicate that our method not only ensures high synthesis quality but also reduces training time. Compared to other algorithms that jointly optimize pose, our training process is sped up by at least 1.3×. In conclusion, our approach efficiently reconstructs scenes under inaccurate poses and offers fresh perspectives and methodologies for pose optimization research in NeRF.

This paper presents preliminary work to utilize a commercial time of flight depth camera for real-time 3D scene reconstruction of underwater objects. Typical RGB stereo camera imaging for 3D capturing suffers from blur and haziness due to turbidity of water in addition to critical dependence on light, either from natural or artificial sources. We propose a method for repurposing the low-cost Microsoft Kinect™ Time of Flight camera for underwater environment enabling dense depth data acquisition that can be processed in real time. Our motivation is the ease of use and low cost of the device for high quality real-time scene reconstruction as compared to multi view stereo cameras, albeit at a smaller range. Preliminary results of depth data acquisition and surface reconstruction in underwater environment are also presented. The novelty of our work is the utilization of the Kinect depth camera for real-time 3D mesh reconstruction and our main objective is to develop an economical and compact solution for underwater 3D mapping.

Real-time 3D reconstruction of static scenes can be achieved based on the RGB-D image sequence fusion. It is a popular practice to divide the space into uniform voxels and use a truncated signed distance function to represent surface information. In order to represent a scene of large scale, the voxel hash algorithm which stores voxels compressively can be used, but most of the conventional methods do not consider the complexity and roughness of the object surface in the scene, so the scene is represented with a uniform resolution. It somewhat limits the range of scene representation and the speed of real-time reconstruction. In this paper, a large-scale scene reconstruction algorithm based on voxel hashing storage with LOD representation is proposed. The main contributions include the following two aspects: (1) By preprocessing the depth image with smooth filtering, which ensures the accuracy of the data, it can effectively reduce the distortion caused by the sensor itself and violent motion and provide better support for the stages of voxel hashing, model rendering, and frame-to-model camera position tracking. (2) The 3D reconstruction with LOD representation is realized. We take the view distance and the roughness of the model surface as criteria to control the adaptive division and representation of spatial voxel blocks. Finally, we carried out qualitative and quantitative evaluations of the algorithm, and confirmed that the algorithm can achieve real-time reconstruction with different levels of detail in the commercial graphics processing hardware environment, and achieve a good fusion effect in large-scale scenes.