Directional Graph Networks with Hard Weight Assignments

Representing valuable semantics in irregular 3-D point clouds with spatial-variant relationships is challenging. Current shared/weighted point convolution methods have limited efficiency and struggle to represent distinctive weight matrices precisely. In thisarticle, we present guided convolution (GuidedConv) operation, a plug-and-play point convolution operator for 3-D point cloud analysis. Compared to existing point convolution methods, GuidedConv implements a guided-share method for the weight matrices, which dynamically assigns corresponding weight matrices to points depending on their local spatial relationships without increasing the kernels in a brute-force way or equipping sophisticated auxiliary networks. The key of GuidedConv is to dynamically assigns weight matrices generated by kernel base to the points according to a guided mask, which is learned adaptively from the spatial relationships of 3-D point clouds in a data-driven manner. Guided masks establish a one-of-a-kind correspondence between the weight matrices and the points. In this way, GuidedConv possesses powerful capabilities to characterize different semantics and is more efficient and precise than current point convolution operators. Extensive qualitative and quantitative evaluations clearly show integrating GuidedConv into several relatively typical networks without modifying configurations to achieve on-par or better performances than state-of-the-art point cloud classification and segmentation methods on several benchmark datasets. Thorough ablation studies and visualizations demonstrate the flexibility and effectiveness of GuidedConv.

3-Dimensional reconstruction (3D reconstruction) generates a 3D computer model of a real object or scene from data such as images, it involves many stages and open problems. Existing methods focus on point clouds and reconstructed polygonal mesh within Manhattan-world constrains in urban scenes reconstruction. However, when dealing with structures like steel truss cable-stayed bridges with complex topology (i.e., connectivity and genus), existing methods fail to recover an appealing polygonal mesh from highly unstructured and noisy point clouds. A topology-aware 3D reconstruction method which can obtain high-level structures and low-level shapes is proposed in this paper. A convolutional neural network and point cloud network is designed to encode multi-view images and 3D point cloud into a compact code, which is then decoded into structure layouts (i.e., a hierarchical binary structural parsing tree) and 3D shapes (i.e., leaf nodes on the binary tree) by designing a recursive neural network and a distance field network respectively. These high-level structures and low-level shapes constitute a 3D digital model.

GrainPointNet: A deep-learning framework for non-invasive sorghum panicle grain count phenotyping

Direct immersogeometric fluid flow and heat transfer analysis of objects represented by point clouds

This paper presents a standardized quality criteria to evaluate the 3D point cloud model of the indoor building which is based on point cloud's data accuracy, the prior characteristics of the building and the coincidence errors of the point cloud model. Our assessment framework involves three steps: the point cloud data acquisition, model generation and quality evaluation. In model generation progress, incapacity of scanning the whole building information one time since its multi-storied spatial structure, the building model need to be registered and merged. In evaluation step, taking into account the need of mapping, indoor location and navigation, the establishment of an interior spatial building model requires accurate measurement. Therefore, we adopt data noise analysis to give a judgement. Then, since the geometric characteristics of the building model are varying, the geometric analysis is proposed to evaluate acquisition errors and registration error. Comparative experiments demonstrate our method give integrate, realistic and reliable quality framework for the indoor building point cloud model.

The study of strain effects in thermally-forced rock masses has gathered growing interest from engineering geology researchers in the last decade. In this framework, digital photogrammetry and infrared thermography have become two of the most exploited remote surveying techniques in engineering geology applications because they can provide useful information concerning geomechanical and thermal conditions of these complex natural systems where the mechanical role of joints cannot be neglected. In this paper, a methodology is proposed for generating point clouds of rock masses prone to failure, combining the high geometric accuracy of RGB optical images and the thermal information derived by infrared thermography surveys. Multiple 3D thermal point clouds and a high-resolution RGB point cloud were separately generated and co-registered by acquiring thermograms at different times of the day and in different seasons using commercial software for Structure from Motion and point cloud analysis. Temperature attributes of thermal point clouds were merged with the reference high-resolution optical point cloud to obtain a composite 3D model storing accurate geometric information and multitemporal surface temperature distributions. The quality of merged point clouds was evaluated by comparing temperature distributions derived by 2D thermograms and 3D thermal models, with a view to estimating their accuracy in describing surface thermal fields. Moreover, a preliminary attempt was made to test the feasibility of this approach in investigating the thermal behavior of complex natural systems such as jointed rock masses by analyzing the spatial distribution and temporal evolution of surface temperature ranges under different climatic conditions. The obtained results show that despite the low resolution of the IR sensor, the geometric accuracy and the correspondence between 2D and 3D temperature measurements are high enough to consider 3D thermal point clouds suitable to describe surface temperature distributions and adequate for monitoring purposes of jointed rock mass.

ABSTRACTVisual–Linguistic large models (V‐LLMs) have demonstrated strong semantic understanding capabilities in understanding visual open‐world scenes. However, there is limited research on how to effectively integrate these models with traditional deep learning networks for classify and segment 3D point clouds. This paper explores the joint training mechanisms of V‐VLLMs and Convolutional Neural Networks (CNNs), constructing a pluggable module that enhances 3D object classification and component segmentation using 2D images, 3D point clouds, and language descriptions. The idea of this pluggable module is to employ contrastive learning between the generalized features extracted by large models from text and image features extracted by the CNN to enhance the robustness and understanding accuracy of the original CNN‐based point cloud network. For experiments, we validated the effectiveness and accuracy of the proposed pluggable module using multiple point cloud classification frameworks, based on datasets such as ModelNet40, ScanObjectNN, and ShapeNetCore. The experimental results demonstrate that the multimodal contrastive learning pluggable module significantly enhances performance across various deep learning frameworks, resulting in significant enhancements in classification and segmentation precision. Additionally, for the multiscale feature PointNet++ framework, the proposed multiscale contrastive learning mechanism provides further performance gains. The Code is available at https://github.com/winbigqian/MCL‐3D‐VCNN.

The thesis is going to focus on point clouds analysis, especially on 3D object classification and part segmentation of point clouds. Point clouds analysis of 3D models and 3D scenes attracts extensive research attention recently, not only due to the availability of several large repositories but also because of its wide applications. Point clouds are commonly used in industry, e.g. autonomous vehicles and robotics use LIDAR to collect point clouds of scenes, which works as an auxiliary recognition tool of 3D space. Therefore, proposing realiable and feasible algorithms for point clouds recognition and analysis is significant and benefitial.

Deep learning on point clouds plays a vital role in a wide range of\napplications such as autonomous driving and AR/VR. These applications interact\nwith people in real-time on edge devices and thus require low latency and low\nenergy. Compared to projecting the point cloud to 2D space, directly processing\nthe 3D point cloud yields higher accuracy and lower #MACs. However, the\nextremely sparse nature of point cloud poses challenges to hardware\nacceleration. For example, we need to explicitly determine the nonzero outputs\nand search for the nonzero neighbors (mapping operation), which is unsupported\nin existing accelerators. Furthermore, explicit gather and scatter of sparse\nfeatures are required, resulting in large data movement overhead. In this\npaper, we comprehensively analyze the performance bottleneck of modern point\ncloud networks on CPU/GPU/TPU. To address the challenges, we then present\nPointAcc, a novel point cloud deep learning accelerator. PointAcc maps diverse\nmapping operations onto one versatile ranking-based kernel, streams the sparse\ncomputation with configurable caching, and temporally fuses consecutive dense\nlayers to reduce the memory footprint. Evaluated on 8 point cloud models across\n4 applications, PointAcc achieves 3.7X speedup and 22X energy savings over RTX\n2080Ti GPU. Co-designed with light-weight neural networks, PointAcc rivals the\nprior accelerator Mesorasi by 100X speedup with 9.1% higher accuracy running\nsegmentation on the S3DIS dataset. PointAcc paves the way for efficient point\ncloud recognition.\n

This paper focuses on the analysis and improvement of K-Neighborhood search of point cloud model, local detail extraction of point cloud model and fusion and transition technology between point cloud models. Through the steps of point cloud scaling, RBF surface reconstruction, segmentation of point cloud to form transition region and uniform resampling of transition region, and the transition point cloud are generated, so the point cloud distribution can be uniform and consistent with the density of the point cloud to be fused. By the method of fusion design, various segmentation and combination of different art models are realized, and it is realized in the file management, view management, data management and other modules of CAD. The final test results show that the effectiveness and robustness of this scheme is of great significance to improve the efficiency and quality of art design.

Despite significant advancements in point cloud analysis, reducing energy consumption and improving robustness remain understudied, largely due to the inherent limitations of Convolutional Neural Networks (CNNs). To address this, we take the cue from the primary visual cortex and propose a Dendritic-Connected Continuous-Coupled Neural Network (DC-CCNN), a novel Brain-Inspired Neural Network (BINN) architecture tailored for point cloud analysis. By leveraging the unique characteristics of point clouds, our design combines discrete and continuous encoding, replacing traditional Multilayer Perceptrons (MLPs) with more efficient and robust BINNs. Our approach substantially improves the performance of Brain-Inspired Neural Networks on point analysis tasks and maintaining performance comparable to state-of-the-art methods. Furthermore, DC-CCNN exhibits enhanced robustness against various point cloud deformations and corruptions. Our experimental results demonstrate that DC-CCNN achieves competitive performance on benchmark datasets, making it a promising alternative to traditional deep learning methods for point cloud analysis. With its high efficiency and robustness, DC-CCNN has the potential for widespread adoption in 3D computer vision, robotics, and autonomous systems.

As 3D acquisition technology develops and 3D sensors become increasingly affordable, large quantities of 3D point cloud data are emerging. How to effectively learn and extract the geometric features from these point clouds has become an urgent problem to be solved. The point cloud geometric information is hidden in disordered, unstructured points, making point cloud analysis a very challenging problem. To address this problem, we propose a novel network framework, called Tree Graph Network (TGNet), which can sample, group, and aggregate local geometric features. Specifically, we construct a Tree Graph by explicit rules, which consists of curves extending in all directions in point cloud feature space, and then aggregate the features of the graph through a cross-attention mechanism. In this way, we incorporate more point cloud geometric structure information into the representation of local geometric features, which makes our network perform better. Our model performs well on several basic point clouds processing tasks such as classification, segmentation, and normal estimation, demonstrating the effectiveness and superiority of our network. Furthermore, we provide ablation experiments and visualizations to better understand our network.

Faced with the deficiency of storing the massive unstructured LiDAR point cloud data with local file system, a cloudizing storage method is proposed for unstructured LiDAR point cloud data based on MongoDB database system. Firstly, this paper designs the architecture of cloudizing storage server cluster for LiDAR point cloud data in the light of distributed storage framework of MongoDB. Secondly, this paper puts forward an organization model of point cloud data file chunks, in accordance with MongoDB distributed file system GridFS, based on which the point cloud file chunks' parallel sharding is achieved. Finally, the prototype system is designed and accomplished for point cloud data cloudizing storage, which is used to carry out experiments on accessing the point cloud data in GridFS, and the results approve that the cloudizing storage proposed in this paper performs better than local file system in the accessing point cloud data.

Mobile vision-based autonomous vehicles use video frames from multiple angles to construct a 3D model of their environment. In this paper, we present a post-processing adaptive noise suppression technique to enhance the quality of the computed 3D model. Our near real-time reconstruction algorithm uses each pair of frames to compute the disparities of tracked feature points to translate the distance a feature has traveled within the frame in pixels into real world depth values. As a result these tracked feature points are plotted to form a dense and colorful point cloud. Due to the inevitable small vibrations in the camera and the mismatches within the feature tracking algorithm, the point cloud model contains a significant amount of misplaced points appearing as noise. The proposed noise suppression technique utilizes the spatial information of each point to unify points of similar texture and color into objects while simultaneously removing noise dissociated with any nearby objects. The noise filter combines all the points of similar depth into 2D layers throughout the point cloud model. By applying erosion and dilation techniques we are able to eliminate the unwanted floating points while retaining points of larger objects. To reverse the compression process, we transform the 2D layer back into the 3D model allowing points to return to their original position without the attached noise components. We evaluate the resulting noiseless point cloud by utilizing an unmanned ground vehicle to perform obstacle avoidance tasks. The contribution of the noise suppression technique is measured by evaluating the accuracy of the 3D reconstruction.

BIM is increasingly crucial for the building construction, yet deviations from actual construction limit its construction monitoring applications. Therefore, this paper proposes a BIM‐construction interaction method that integrates parametric modeling, point cloud processing, and parameter optimization and fitting to enhance the construction monitoring. Proposed parametric modeling methods equip BIM elements with the ability of the pose adjustment and deformation modification, addressing potential deviations during construction. Developed component feature extraction algorithms efficiently capture pose and deformation features from the point cloud model that sufficiently and accurately reflect the actual state of the structural components. The proposed parameter optimization and fitting approach targets model parameters for optimization and aims to match pose and deformation features for fitting. By constructing objective functions that quantify the deviation between the BIM and point cloud models, the process is driven by the RBFOpt optimization algorithm and Opossum optimization solver. This approach enables the automatic updating of the design BIM into the as‐built BIM and generates deviation data between the two models, providing a basis for comprehensive construction monitoring results. The BIM‐construction interaction method was applied to the core area construction of the Shanghai Grand Opera House, where it reduced the root mean square error (RMSE) between the parametric BIM and point cloud models of the core column, concrete thick shells, and cantilever beams from 0.0352 m, 0.0411 m, and 0.0323 m to 0.0082 m, 0.0323 m, and 0.0053 m, respectively, significantly reducing deviations between BIM and the actual construction. Comprehensive and quantitative construction monitoring data, including pose deviations and structural deformations, were obtained to assess the precision and safety of the core area construction. The results demonstrate that the BIM‐construction interaction method effectively supports the interaction between BIM and construction, enabling monitoring and evaluation based on point cloud data.