3D SOC-Net: Deep 3D reconstruction network based on self-organizing clustering mapping

In modern medicine, 3D reconstruction plays an important role in improving the accuracy of disease detection and treatment especially in the process of robot-assisted surgery. Accordingly, the choice of 3D reconstruction method is of great importance, which provides real-time visualization. This study analyses and compares two types of methods in 3D medical image reconstruction and visualization: Surface Rendering and Volume Rendering. To be more specific, four algorithms including Contour Filter (CF), Marching Cubes (MC), Composite Volume Rendering (CVR) and Texture Mapping Hardware (TMH) were implemented and compared from the perspective of the rendering effect, speed and interactivity, in which the former two algorithms belong to the concept of Surface Rendering and the latter two algorithms are in the range of Volume Rendering. These algorithms were implemented by Microsoft Visual Studio with Visualization Toolkit (VTK) to reconstruct the 3D images of patients CT image data. The mechanism and realization of these algorithms were described, and the merits and shortcomings of these different rendering methods were compared. Based on the comparison results, the 3D reconstruction and rendering method can be determined according to the specific requirements of robot-assisted surgery.

This work deals with the automatic 3D reconstruction of objects from frontal RGB images. This aims at a better understanding of the reconstruction of 3D objects from RGB images and their use in immersive virtual environments. We propose a complete workflow that can be easily adapted to almost any other family of rigid objects. To explain and validate our method, we focus on guitars. First, we detect and segment the guitars present in the image using semantic segmentation methods based on convolutional neural networks. In a second step, we perform the final 3D reconstruction of the guitar by warping the rendered depth maps of a fitted 3D template in 2D image space to match the input silhouette. We validated our method by obtaining guitar reconstructions from real input images and renders of all guitar models available in the ShapeNet database. Numerical results for different object families were obtained by computing standard mesh evaluation metrics such as Intersection over Union, Chamfer Distance, and the F-score. The results of this study show that our method can automatically generate high-quality 3D object reconstructions from frontal images using various segmentation and 3D reconstruction techniques.

3D vehicle shape completion is a key task in urban environment reconstruction, since the available sensors and 3D scanning procedures (such as Mobile Laser Scanning) in an outdoor city scene cannot usually extract the entire car shapes, which can be necessary for specifying their geometric properties for further analysis or realistic visualization. In this paper, we propose a novel multi-view based 3D object point cloud completion technique. In contrast to existing approaches, our method operates on 2D images formed by projecting the point cloud from several virtual camera positions around the object of interest. Both color and geometrical information is considered during the process, generating dense textured point clouds, displaying realistic patterns in the missing regions from the partial inputs. We present both quantitative and qualitative tests on various synthetic and real laser scanned vehicle point clouds, which demonstrate that our method surpasses existing state-of-the-art approaches. By applying it to vehicles from the Shapenet dataset, our approach outperforms recent techniques in terms of Earth Mover’s Distance (EMD) and Chamfer Distance (CD) by 43.8% and 12.17%, respectively.

The Earth Mover''s Distance (EMD) between two finite distributions of weight is proportional to the minimum amount of work required to transform one distribution into the other. Current content-based retrieval work in the Stanford Vision Laboratory uses the EMD as a common framework for measuring image similarity with respect to color, texture, and shape content. In this report, we present some fast to compute lower bounds on the EMD which may allow a system to avoid exact, more expensive EMD computations during query processing. The effectiveness of the lower bounds is tested in a color-based retrieval system. In addition to the lower bound work, we also show how to compute the EMD under translation. In this problem, the points in one distribution are free to translate, and the goal is to find a translation that minimizes the EMD to the other distribution.

Effective measurement of the reflective or transparent surface of an object has always been a disadvantage in laser scanning modeling. We propose a fast and complete three-dimensional (3D) reconstruction method for small static objects using laser scanning and the structure from motion (SFM) algorithm. Meanwhile, a complete reconstruction workflow is designed and a multi-angle 3d reconstruction system is set up. To generate the complete point cloud model of the object, the SFM algorithm is used to reconstruct the surface part of the object, the data for which cannot be obtained by the laser measuring instrument. The experimental results show that this method not only improves the speed, accuracy, integrity, and visual effect of 3D reconstruction of small objects, but also extends the scope of 3D reconstruction of laser measurement.

Earth Mover's Distance (EMD), as a similarity measure, has received a lot of attention in the fields of multimedia and probabilistic databases, computer vision, image retrieval, machine learning, etc. EMD on multidimensional histograms provides better distinguishability between the objects approximated by the histograms (e.g., images), compared to classic measures like Euclidean distance. Despite its usefulness, EMD has a high computational cost; therefore, a number of effective filtering methods have been proposed, to reduce the pairs of histograms for which the exact EMD has to be computed, during similarity search. Still, EMD calculations in the refinement step remain the bottleneck of the whole similarity search process. In this paper, we focus on optimizing the refinement phase of EMD-based similarity search by (i) adapting an efficient min-cost flow algorithm (SIA) for EMD computation, (ii) proposing a dynamic distance bound, which can be used to terminate an EMD refinement early, and (iii) proposing a dynamic refinement order for the candidates which, paired with a concurrent EMD refinement strategy, reduces the amount of needless computations. Our proposed techniques are orthogonal to and can be easily integrated with the state-of-the-art filtering techniques, reducing the cost of EMD-based similarity queries by orders of magnitude.

With the increasing use of image-based 3D reconstruction in medical procedures, accurate scene reconstruction plays a crucial role in surgical navigation and assisted treatment. However, the monotonous colors, limited image features, and obvious brightness fluctuations of thoracoscopic scenes make the feature point matching process, on which traditional 3D reconstruction methods rely, unstable and unreliable. It brings a great challenge to accurate 3D reconstruction. In this study, a new method for implicit 3D reconstruction of monocular thoracoscopic scenes is proposed. The method combines a pre-trained metric depth estimation model with neural radiation field (NeRF) technique and uses dense SLAM to accurately compute the camera pose. To ensure the accuracy of the depth values and the structural consistency of the reconstructed scene, depth and normal constraints are added to the original color constraints of the NeRF network to achieve high-quality scene reconstruction results. We conducted experiments on the SCARED dataset and the clinical dataset. After comparing with other methods, the depth estimation accuracy and point cloud reconstruction quality of this paper outperform the existing methods. The method in this paper can provide more accurate 3D reconstruction of complex thoracic surgical scenes, which can significantly improve the accuracy and therapeutic efficacy of surgical navigation.

With the development of computer vision technologies, 3D reconstruction has become a hotspot. At present, 3D reconstruction relies heavily on expensive equipment and has poor real-time performance. In this paper, we aim at solving the problem of 3D reconstruction of an indoor scene with large vertical span. In this paper, we propose a novel approach for 3D reconstruction of indoor scenes with only a Kinect. Firstly, this method uses a Kinect sensor to get color images and depth images of an indoor scene. Secondly, the combination of scale-invariant feature transform and random sample consensus algorithm is used to determine the transformation matrix of adjacent frames, which can be seen as the initial value of iterative closest point (ICP). Thirdly, we establish the relative coordinate relation between pair-wise frames which are the initial point cloud data by using ICP. Finally, we achieve the 3D visual reconstruction model of indoor scene by the top-down image registration of point cloud data. This approach not only mitigates the sensor perspective restriction and achieves the indoor scene reconstruction of large vertical span, but also develops the fast algorithm of indoor scene reconstruction with large amount of cloud data. The experimental results show that the proposed algorithm has better accuracy, better reconstruction effect, and less running time for point cloud registration. In addition, the proposed method has great potential applied to 3D simultaneous location and mapping.

Supplier selection is a critical decision-making process in supply chain management, significantly impacting operational efficiency and cost-effectiveness. Traditional methods often struggle to handle the inherent uncertainty and vagueness involved in evaluating supplier criteria. This paper proposes a novel approach for supplier selection using Intuitionistic Trapezoidal Fuzzy Numbers (ITFNs), combined with Earth Mover's Distance (EMD) and Fuzzy Technique for Order Preference by Similarity to Ideal Solution (Fuzzy TOPSIS), to effectively model and address these uncertainties. ITFNs allow for the representation of both membership and non-membership degrees, enhancing the ability to capture imprecision in supplier evaluations. The Earth Mover's Distance, a powerful metric from transportation theory, is employed to rank the intuitionistic trapezoidal fuzzy numbers and compare suppliers based on multiple criteria using fuzzy TOPSIS. The proposed model is demonstrated through a case study, showcasing its ability to provide more flexible and accurate supplier rankings compared to conventional methods. This approach offers valuable insights for decision-makers aiming to optimize supplier selection in complex and uncertain environments, contributing to more resilient supply chains.

Multi-view 3D reconstruction aims to discover 3D characteristics based on visual information captured across multiple viewpoints. Transformer networks have shown remarkable success in various computer vision tasks, including multi-view 3D reconstruction. However, the reconstruction of accurate 3D shapes faces challenges when trying to efficiently extract and merge features across views. The existing frameworks struggled to capture the subtle relationships between the views, resulting in a poor reconstruction. To address this issue, we present a new framework, WGA-SWIN, for 3D reconstruction using multi-view objects. Our method introduces a Window Grouping Attention (WGA) mechanism that uses group tokens from different views for each window attention operation, enabling efficient inter-view and intra-view feature extraction. Diversity among various groups in a model contributes to the richness of feature learning, which results in advanced and dependable feature learning, resulting in more comprehensive and robust representations. Within the encoder swin transformer blocks, we integrated WGA to utilize both hierarchical design and shifted window attention mechanisms for efficient multi-view feature extraction. In addition, we developed a progressive hierarchical decoder that combines swin transformer blocks with 3D convolutions to utilize voxel representation, resulting in a high resolution for obtaining high-quality 3D reconstructions with fine structural details. The experimental results on the benchmark datasets ShapeNet and Pix3D demonstrate that our work achieves state-of-the-art (SOTA) performance, outperforming existing methods in both single-view and multi-view 3D reconstruction, beyond the capabilities of current technologies. We lead by 0.95% and 1.07% in both IoU and F-Scores respectively, which demonstrates the robustness of our method.

Real-time three-dimensional (3D) reconstruction of real-world environments has many significant applications in various fields, including telepresence technology. When depth sensors, such as those from Microsoft's Kinect series, are introduced simultaneously and become widely available, a new generation of telepresence systems can be developed by combining a real-time 3D reconstruction method with these new technologies. This combination enables users to engage with a remote person while remaining in their local area, as well as control remote devices while viewing their 3D virtual representation. There are numerous applications in which having a telepresence experience could be beneficial, including remote collaboration and entertainment, as well as education, advertising, and rehabilitation. The purpose of this systematic literature review is to analyze the recent advances in 3D reconstruction methods for telepresence systems and the significant related work in this field. Next, we determine the input data and the technological device employed to acquire the input data, which will be utilized in the 3D reconstruction process. The methods of 3D reconstruction implemented in the telepresence system as well as the evaluation of the system, have been extracted and assessed from the included studies. Through the analysis and summarization of many dimensions, we discussed the input data used for the 3D reconstruction method, the real-time 3D reconstruction methods implemented in the telepresence system, and how to evaluate the system. We conclude that real-time 3D reconstruction methods for telepresence systems have progressively improved over the years in conjunction with the advancement of machines and devices such as Red Green Blue-Depth (RGB-D) cameras and Graphics Processing Unit (GPU).

As the basis of intelligent breeding management and animal husbandry insurance, the identification of individual cattle is important in animal husbandry management. Given the difficulty of data acquisition caused by the non-rigid and lacking cooperation of cattle, this study proposes a method for cattle face image acquisition and processing that can efficiently adapt to the harsh environment of cattle barns. When processing the non-rigid cow face, the method of approximating the cow face to a rigid body is used to establish the cow face image data set., and the cattle face image data set is established. The Three Dimensional(3D) reconstruction method of cattle face uses a 3D image reconstruction method based on multiple perspectives. First, the scale-invariant feature transform algorithm is used to extract the image feature points. The fast library for approximate nearest neighbors algorithm is used to match feature points. The matching results are selected via random sampling consensus. Second, the structure of the motion method is used for the sparse reconstruction of point clouds, and the dense point cloud is then generated using the three-dimensional multi-view stereo vision algorithm. Finally, the Poisson surface reconstruction method is used for surface reconstruction. The results indicate that this method can effectively realize the three-dimensional reconstruction of cattle faces; the reconstructed images have obvious color, clear texture, and complete shape features.

A novel approach for assessing the extent of electron displacement in optical transitions is proposed by implementing the Earth Mover's Distance (EMD) method, which quantifies the spatial dissimilarity between ground and excited state electron density distributions. In contrast to previous descriptors, this index provides a representative and intuitively understandable distance under a robust and computationally efficient scheme for all possible forms of locality, even in the most difficult to dissect topological cases. The theoretical differences among the existing indices and our method are first illustrated with the help of a simplified model system, followed by a benchmarking of several partial atomic charge models using experimentally relevant push-pull compounds with diverse symmetries. These same molecules are finally employed to further demonstrate the principal advantages of the EMD index and its capabilities in rationalizing charge transfer phenomena.

Color descriptors are one of the important features used in content-based image retrieval. The Dominant Color Descriptor (DCD) represents a few perceptually dominant colors in an image through color quantization. For image retrieval based on DCD, the earth mover's distance and the optimal color composition distance are proposed to measure the dissimilarity between two images. Although providing good retrieval results, both methods are too time-consuming to be used in a large image database. To solve the problem, we propose a new distance function that calculates an approximate earth mover's distance in linear time. To calculate the dissimilarity in linear time, the proposed approach employs the space-filling curve for multidimensional color space. To improve the accuracy, the proposed approach uses multiple curves and adjusts the color positions. As a result, our approach achieves order-of-magnitude time improvement but incurs small errors. We have performed extensive experiments to show the effectiveness and efficiency of the proposed approach. The results reveal that our approach achieves almost the same results with the EMD in linear time.

In this paper, we propose a novel approach to learning robust ground distance functions of the Earth Mover's distance to make it appropriate for quantifying the partial similarity between two feature-sets. First, we define the ground distance as a monotonic transformation of commonly used feature-to-feature base distance (or similarity) measures, so that in computing the Earth Mover's distance, the algorithm could better turn its focus on the feature pairs that are correctly matched, while being less affected by irrelevant ones. As a result, the proposed method is especially suited for 3D partial shape retrieval where occlusion and clutter are serious problems. We prove that when the transformation satisfies certain conditions, the metric property of the base distance is sufficient to guarantee the ground distance is a metric (and so is the Earth Mover's distance), which makes fast shape retrieval on large databases technically possible. Second, we propose a discriminative learning framework to optimize the transformation function based on the real Adaboost algorithm. The optimization is performed in the space of the piecewise constant approximations of the transformation without making any parametric assumption. Finally, extensive experiments on 3D partial shape retrieval convincingly demonstrate the effectiveness of the proposed techniques.