Feature Extraction from 3D Point Cloud Data Based on Discrete Curves

This work deals with development of algorithm for physical replication of patient specific human bone and construction of corresponding implants/inserts RP models by using Reverse Engineering approach from non-invasive medical images for surgical purpose. In medical field, the volumetric data i.e. voxel and triangular facet based models are primarily used for bio-modelling and visualization, which requires huge memory space. On the other side, recent advances in Computer Aided Design (CAD) technology provides additional facilities/functions for design, prototyping and manufacturing of any object having freeform surfaces based on boundary representation techniques. This work presents a process to physical replication of 3D rapid prototyping (RP) physical models of human bone from various CAD modeling techniques developed by using 3D point cloud data which is obtained from non-invasive CT/MRI scans in DICOM 3.0 format. This point cloud data is used for construction of 3D CAD model by fitting B-spline curves through these points and then fitting surface between these curve networks by using swept blend techniques. This process also can be achieved by generating the triangular mesh directly from 3D point cloud data without developing any surface model using any commercial CAD software. The generated STL file from 3D point cloud data is used as a basic input for RP process. The Delaunay tetrahedralization approach is used to process the 3D point cloud data to obtain STL file. CT scan data of Metacarpus (human bone) is used as the case study for the generation of the 3D RP model. A 3D physical model of the human bone is generated on rapid prototyping machine and its virtual reality model is presented for visualization. The generated CAD model by different techniques is compared for the accuracy and reliability. The results of this research work are assessed for clinical reliability in replication of human bone in medical field.

Building Information Modeling (BIM) has gained wider acceptance in the Architecture/Engineering/Construction (A/E/C) industries in the U.S. and internationally. Capability of BIM technologies in early detection of spatial conflicts among various building systems and components concurrently designed by different design team members prior to construction has great potential in reducing construction delays, loss of productivity, and overall construction costs. Recent advancements in 3D point cloud technologies, such as 3D laser scanning, can greatly augment BIM technology applications in the A/E/C industry by providing a rapid three-dimensional capture of the built environment in great detail and accuracy. This paper discusses the current approaches of implementing 3D point cloud data in BIM and virtual design and construction (VDC) applications during various stages of a project lifecycle and the challenges associated with processing the huge amount of 3D point cloud data. Conversion from discrete 3D point cloud raster data to geometric/vector BIM data remains to be a labor-intensive process. The needs for intelligent geometric feature detection/reconstruction algorithms for automated point cloud processing and issues related to data management will be discussed. This paper also presents some innovative approaches for integrating 3D point cloud data with BIM to efficiently augment built environment design, construction, and management.

The objective of this work is to develop and apply a spectral graph theoretic approach for differentiating between (classifying) additive manufactured (AM) parts contingent on the severity of their dimensional variation from laser-scanned coordinate measurements (3D point cloud). The novelty of the approach is in invoking spectral graph Laplacian eigenvalues as an extracted feature from the laser-scanned 3D point cloud data in conjunction with various machine learning techniques. The outcome is a new method that classifies the dimensional variation of an AM part by sampling less than 5% of the 2 million 3D point cloud data acquired (per part). This is a practically important result, because it reduces the measurement burden for postprocess quality assurance in AM—parts can be laser-scanned and their dimensional variation quickly assessed on the shop floor. To realize the research objective, the procedure is as follows. Test parts are made using the fused filament fabrication (FFF) polymer AM process. The FFF process conditions are varied per a phased design of experiments plan to produce parts with distinctive dimensional variations. Subsequently, each test part is laser scanned and 3D point cloud data are acquired. To classify the dimensional variation among parts, Laplacian eigenvalues are extracted from the 3D point cloud data and used as features within different machine learning approaches. Six machine learning approaches are juxtaposed: sparse representation, k-nearest neighbors, neural network, naïve Bayes, support vector machine, and decision tree. Of these, the sparse representation technique provides the highest classification accuracy (F-score > 97%).

Information of unordered point cloud is limited because of no direct topologic relation between points or triangular facets. So it will be difficult to obtain the feature points of 3D point cloud data. In this article, we use the geometry properties, such as normal, curvature and density of the points' information to detect features of the 3D point cloud data and propose a curvature and density based feature point detection method for unordered 3D point cloud data. Firstly, we define a feature parameter of 3D point cloud data, which includes the distance with its neighboring points, the sum of the normal angle between the point and neighboring points, and point cloud data curvature. Secondly, the density of data points is calculated by using Octree and is used as the features of points by a threshold of their feature parameter. The experimental results show that our new approach might detect feature points accurately for the given 3D point cloud data.

Robots typically possess sensors of different modalities, such as colour cameras, inertial measurement units, and 3D laser scanners. Often, solving a particular problem becomes easier when more than one modality is used. However, while there are undeniable benefits to combine sensors of different modalities the process tends to be complicated. Segmenting scenes observed by the robot into a discrete set of classes is a central requirement for autonomy as understanding the scene is the first step to reason about future situations. Scene segmentation is commonly performed using either image data or 3D point cloud data. In computer vision many successful methods for scene segmentation are based on conditional random fields (CRF) where the maximum a posteriori (MAP) solution to the segmentation can be obtained by inference. In this paper we devise a new CRF inference method for scene segmentation that incorporates global constraints, enforcing the sets of nodes are assigned the same class label. To do this efficiently, the CRF is formulated as a relaxed quadratic program whose MAP solution is found using a gradient-based optimisation approach. The proposed method is evaluated on images and 3D point cloud data gathered in urban environments where image data provides the appearance features needed by the CRF, while the 3D point cloud data provides global spatial constraints over sets of nodes. Comparisons with belief propagation, conventional quadratic programming relaxation, and higher order potential CRF show the benefits of the proposed method.

Abstract. The automatic analysis of large 3D point clouds represents a crucial task in photogrammetry, remote sensing and computer vision. In this paper, we propose a new methodology for the semantic interpretation of such point clouds which involves feature relevance assessment in order to reduce both processing time and memory consumption. Given a standard benchmark dataset with 1.3 million 3D points, we first extract a set of 21 geometric 3D and 2D features. Subsequently, we apply a classifier-independent ranking procedure which involves a general relevance metric in order to derive compact and robust subsets of versatile features which are generally applicable for a large variety of subsequent tasks. This metric is based on 7 different feature selection strategies and thus addresses different intrinsic properties of the given data. For the example of semantically interpreting 3D point cloud data, we demonstrate the great potential of smaller subsets consisting of only the most relevant features with 4 different state-of-the-art classifiers. The results reveal that, instead of including as many features as possible in order to compensate for lack of knowledge, a crucial task such as scene interpretation can be carried out with only few versatile features and even improved accuracy.

Development of vehicle- recognition method on water surfaces using LiDAR data: SPD2 (spherically stratified point projection with diameter and distance)

Industrial anomaly detection by integrating RGB and 3D point cloud data has demonstrated significant potential. However, existing methods predominantly rely on 3D data, underutilizing the complementary value of RGB information. To address this, we propose a dual-branch framework that processes 2D and 3D modalities separately. In the 2D branch, we design an enhanced single-step guided diffusion model. Unlike conventional approaches that solely predict noise for reconstruction, we argue that anomalies and noise are fundamentally distinct, and explicit modeling of anomalous features is essential for effective restoration. We innovatively define the network objective as the simultaneous prediction of both noise and anomaly components. By injecting synthetic anomalies into normal samples during training, the network learns the joint distribution of "noise-anomaly". This design enables the model to more accurately identify and remove anomalies while better restoring normal structures, significantly improving reconstruction quality. To maintain efficiency, we introduce a single-step denoising mechanism based on a normalized guidance strategy, where coarser noise scales guide reconstruction at finer scales, achieving improved reconstruction accuracy while substantially accelerating inference.In the 3D branch, we build upon the 3DSR baseline and integrate Structural Similarity Index (SSIM) loss to enhance structure preservation, particularly improving detail and edge recovery, thereby increasing detection accuracy and robustness. Furthermore, we propose a dynamic learning factor strategy to optimize training convergence and stability. Finally, the outputs of both branches are fused using a One-Class Support Vector Machine (SVM) for more reliable anomaly discrimination. Our method achieves state-of-the-art performance on the MVTec 3D dataset, attaining 99.5% image-level AUROC and 99.5% pixel-level AUROC, outperforming existing mainstream approaches. The code is available at: https://github.com/cz123456-china/Diff3d-ad/.

Autonomous aerial refueling (AAR) is an important capability for an unmanned aerial vehicle (UAV) to increase its flying range and endurance without increasing its size. This paper presents a novel tracking method that utilizes both 2D intensity and 3D point-cloud data acquired with a 3D Flash LIDAR sensor to establish relative position and orientation between the receiver vehicle and drogue during an aerial refueling process. Unlike classic, vision-based sensors, a 3D Flash LIDAR sensor can provide 3D point-cloud data in real time without motion blur, in the day or night, and is capable of imaging through fog and clouds. The proposed method segments out the drogue through 2D analysis and estimates the center of the drogue from 3D point-cloud data for flight trajectory determination. A level-set front propagation routine is first employed to identify the target of interest and establish its silhouette information. Sufficient domain knowledge, such as the size of the drogue and the expected operable distance, is integrated into our approach to quickly eliminate unlikely target candidates. A statistical analysis along with a random sample consensus (RANSAC) is performed on the target to reduce noise and estimate the center of the drogue after all 3D points on the drogue are identified. The estimated center and drogue silhouette serve as the seed points to efficiently locate the target in the next frame.

Smart Merging Method for Hybrid Point Cloud Data using UAV and LIDAR in Earthwork Construction

To detect the cone of the Formula Student Autonomous China (FSAC) track quickly and accurately, this paper proposed a fast and accurate perception system that fused the 3D LIDAR point cloud data and visual information. First, the vision part spliced the images of the two monocular cameras vertically and adopted the YOLOv3 target detection network with the attention mechanism to obtain the type of candidate targets and their position on the pixel plane. Afterward, the track boundary could be extracted and the drivable area could be segmented using cubic polynomial regression by the center of the bounding box. The LIDAR pipeline used the GPF algorithm and the Euclidean clustering algorithm to process the original point cloud to obtain the three-dimensional information of the cones’ coordinates. Finally, the 3D LIDAR point cloud data of reconstructed cones was projected onto the two-dimensional pixel plane and whether the target was detected was determined by judging whether the projected point was in a specific position within the bounding box. Experimental results showed that the proposed method had a larger perception range, faster detection speed, and higher detection accuracy compared with the 2020 season's perception system.

3D point cloud data captured from Oil and Gas (O&G) facilities are becoming increasingly abundant. This data supports the interactive visualization of complex plants, which is central to numerous industrial advancements. As industries seek to innovate and optimize their processes, the versatile use of 3D point cloud data is proving invaluable beyond traditional O&G applications. The precision and comprehensiveness of 3D point cloud data facilitate efficient design, construction, maintenance and operational monitoring across various industrial sectors. Digital Twin (DT) solutions enable this by maintaining 3D representations, such as 3D CAD models, that accurately reflect the current state of facilities. As installations change, the process of 3D scanning is often repeated to capture the latest updates. Engineers and designers then use modelling software to visualize the resulting point cloud and update the facility's 3D representation, incorporating any new modifications. However, this involves interpreting the visual point cloud data and structuring it into meaningful objects (such as valves, tanks, etc.), a process that remains manual, requiring labor-intensive tasks that are prone to errors and incur significant costs due to the need for specialized labor. We propose an Artificial Intelligence approach, that uses a deep learning-based algorithm to automatically segment 3D objects within a point cloud. Our workflow recognizes instances of usual components of production systems, such as control valves, pipes, and transmitters. A state-of-the-art 3D instance segmentation approach is adapted to the O&G domain. Given the lack of publicly available instance segmentation datasets in this domain, we manually annotate 54 scenes from an O&G offshore plant, each of them around 20 m2 in footprint. To further enhance model training, we also incorporated synthetic point clouds. Practical results on real point clouds, acquired from operational facilities and pre-processed as required by commercial DT solutions, indicated a performance of approximately 70% success (f1-score) in segmenting components of O&G production systems, reaching up to 85% for components such as control valves. Accordingly, a 52.4% reduction in costs and efforts associated with manually interpreting 3D point clouds is estimated. Point cloud instance segmentation has attracted much attention in the computer vision research community, and many approaches have been proposed. However, results in the literature are heavily biased toward a few benchmarking datasets, which were acquired in indoor environments like offices and rooms. Few studies have addressed the O&G domain, creating a gap in the literature for achieving more competitive results. This lack of comprehensive research limits the potential for innovation and optimization in the industry. This paper contributes by (i) assessing the performance of state-of-the-art algorithms in the domain of O&G facilities, (ii) proposing modifications to better generalize to components of O&G production systems, (iii) objectively measuring the impact of this algorithm in reducing manual effort for engineers, and (iv) advancing towards a new DT functionality that derives an asset 3D representation, that is faithful to physical reality, through the automatic interpretation of point cloud scans.

Three-dimensional (3D) semantic segmentation based on point clouds has become a critical technology in the field of intelligent spatial perception and scene understanding. However, most existing methods cannot fully exploit the spatial feature of 3D point cloud data and the orderliness of two-dimensional (2D) image information, resulting in an inability to effectively improve segmentation accuracy. This study proposes a novel 2D image and 3D point cloud fusion method named shared multi-layer perceptron fusion semantic segmentation (SMFusionSeg) for semantic segmentation. The rich features of 2D image segmentation and spatial information of 3D point clouds are integrated by designing a fusion architecture that facilitates the transfer of 2D semantics to 3D semantics. Specifically, the fusion architecture concatenates the features from both 2D and 3D domains, which are then further integrated through a shared multi-layer perceptron (MLP) and optimized using an attention mechanism to enhance feature effectiveness and distinctiveness. Moreover, a knowledge distillation strategy is introduced to improve the learning capability and segmentation accuracy in parsing complex scenes. Experimental results on the SemanticKITTI dataset demonstrate that the proposed SMFusionSeg achieves a mean Intersection over Union ( mIoU ) of 65.4%, significantly outperforming traditional single-modal methods and many existing segmentation techniques, thereby effectively enhancing the precision and robustness of geographic feature recognition.

An imaging sonar loses information of elevation angle while a mapping process. To overcome this limitation, the motion of the autonomous underwater vehicle (AUV) can be used to obtain 3D information using the imaging sonar. In this paper, we propose a two-stage mapping strategy for accurately generating underwater 3D maps based on an imaging sonar. It consists of searching and scanning stage. In the scanning stage, multi-directional scanning is performed on an object. To process 3D point cloud data obtained by multi-directional scanning, we propose a polygonal approximation method. This method reduces the uncertainty of 3D point cloud data by extracting intersection area of multiple data groups. To verify the feasibility of proposed strategies, we conducted indoor tank experiments using a hovering-type AUV `Cyclops' and acoustic lens-based multibeam sonar (ALMS) `DIDSON'.

Recently, light detection and ranging (LiDAR) technology has emerged as a promising approach to enhance sensing systems, particularly in human activity recognition (HAR). Advances in computer vision and LiDAR sensor development are revolutionizing HAR systems. This work proposes a novel method to detect and classify human exercise poses using 3D LiDAR technology by taking advantage of deep learning techniques. We utilize the Point Pillars network for 3D object detection and employ pseudo-image transformation for effective 3D to 2D data conversion, enhancing the applicability of 2D CNNs for 3D point cloud data analysis. To account for temporal dependencies in 3D point cloud data streams, we use the "smush" technique, which considers a few consecutive 3D LiDAR point cloud frames. This method significantly improves the detection accuracy of dynamic exercises, highlighting the potential of our approach in enhancing fitness monitoring technologies. This research not only demonstrates the effective use of 3D LiDAR in capturing complex activities but also lays the groundwork for future advancements in mapping human movement using 3D LiDAR. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>