Automatic Recognition of Machining Features Based on Point Cloud Data Using Convolution Neural Networks

Face recognition is a challenging task as it has to deal with several issues such as illumination orientation and variability among the different faces. Previous works have shown that 3D face is a robust biometric trait and is less sensitive to light and pose variations. Also due to availability of inexpensive sensors and new 3D data acquisition techniques it has become easy to capture 3D data. A 3D depth image of a face is found to be rich in information and biometric recognition performance can be enhanced by using 3D face data along with convolutional neural network. However the shortcoming of this approach is the conversion of 3D data to lower dimensions (depth image) which suffer from loss of geometric information and the network becomes computationally expensive. In this work we endeavor to apply deep learning method for 3D face recognition and propose a deep convolutional neural network based on PointNet architecture which consumes point cloud directly as input and siamese network for similarity learning. Further we propose a solution to the issue of a limited database by applying data augmentation at the point cloud level. Our proposed technique shows encouraging performance on Bosphorus and IIT Indore 3D face databases.

A 3D surface modeling format that is commonly used today is point cloud. 3D surface model segmentation could provide data for analysis in various fields. In the context of Geographic Information Systems, point cloud data obtained from the Light Detection and Ranging (LiDAR) sensor are used by machines to automatically identify objects such as houses, buildings, land, and rivers. There has been many Deep Learning approach through Convolutional Neural Network (CNN) that has been proven to be very capable for 2-dimensional imagery classification and segmentation. PointNet is a Deep Learning architecture that is designed so that the point cloud format that is still tabular form, can be directly convoluted by the CNN model. In this study, an improvement of PointNet is proposed for Point Cloud data of Kupang City. The Point Cloud data were acquired using an Unmanned Aerial Vehicle with a LiDAR sensor installed. The data were pre-processed and divided into training and testing data. The data were processed with the PointNet architecture and the model was tested using several metrics. The experiment shows that the PointNet architecture is capable on segmenting Geographical Point Cloud Data. In addition, incorporating voxel's color features could increase the performance of the segmentation.

Development of vehicle- recognition method on water surfaces using LiDAR data: SPD2 (spherically stratified point projection with diameter and distance)

This work deals with development of algorithm for physical replication of patient specific human bone and construction of corresponding implants/inserts RP models by using Reverse Engineering approach from non-invasive medical images for surgical purpose. In medical field, the volumetric data i.e. voxel and triangular facet based models are primarily used for bio-modelling and visualization, which requires huge memory space. On the other side, recent advances in Computer Aided Design (CAD) technology provides additional facilities/functions for design, prototyping and manufacturing of any object having freeform surfaces based on boundary representation techniques. This work presents a process to physical replication of 3D rapid prototyping (RP) physical models of human bone from various CAD modeling techniques developed by using 3D point cloud data which is obtained from non-invasive CT/MRI scans in DICOM 3.0 format. This point cloud data is used for construction of 3D CAD model by fitting B-spline curves through these points and then fitting surface between these curve networks by using swept blend techniques. This process also can be achieved by generating the triangular mesh directly from 3D point cloud data without developing any surface model using any commercial CAD software. The generated STL file from 3D point cloud data is used as a basic input for RP process. The Delaunay tetrahedralization approach is used to process the 3D point cloud data to obtain STL file. CT scan data of Metacarpus (human bone) is used as the case study for the generation of the 3D RP model. A 3D physical model of the human bone is generated on rapid prototyping machine and its virtual reality model is presented for visualization. The generated CAD model by different techniques is compared for the accuracy and reliability. The results of this research work are assessed for clinical reliability in replication of human bone in medical field.

Information of unordered point cloud is limited because of no direct topologic relation between points or triangular facets. So it will be difficult to obtain the feature points of 3D point cloud data. In this article, we use the geometry properties, such as normal, curvature and density of the points' information to detect features of the 3D point cloud data and propose a curvature and density based feature point detection method for unordered 3D point cloud data. Firstly, we define a feature parameter of 3D point cloud data, which includes the distance with its neighboring points, the sum of the normal angle between the point and neighboring points, and point cloud data curvature. Secondly, the density of data points is calculated by using Octree and is used as the features of points by a threshold of their feature parameter. The experimental results show that our new approach might detect feature points accurately for the given 3D point cloud data.

3D point cloud data captured from Oil and Gas (O&G) facilities are becoming increasingly abundant. This data supports the interactive visualization of complex plants, which is central to numerous industrial advancements. As industries seek to innovate and optimize their processes, the versatile use of 3D point cloud data is proving invaluable beyond traditional O&G applications. The precision and comprehensiveness of 3D point cloud data facilitate efficient design, construction, maintenance and operational monitoring across various industrial sectors. Digital Twin (DT) solutions enable this by maintaining 3D representations, such as 3D CAD models, that accurately reflect the current state of facilities. As installations change, the process of 3D scanning is often repeated to capture the latest updates. Engineers and designers then use modelling software to visualize the resulting point cloud and update the facility's 3D representation, incorporating any new modifications. However, this involves interpreting the visual point cloud data and structuring it into meaningful objects (such as valves, tanks, etc.), a process that remains manual, requiring labor-intensive tasks that are prone to errors and incur significant costs due to the need for specialized labor. We propose an Artificial Intelligence approach, that uses a deep learning-based algorithm to automatically segment 3D objects within a point cloud. Our workflow recognizes instances of usual components of production systems, such as control valves, pipes, and transmitters. A state-of-the-art 3D instance segmentation approach is adapted to the O&G domain. Given the lack of publicly available instance segmentation datasets in this domain, we manually annotate 54 scenes from an O&G offshore plant, each of them around 20 m2 in footprint. To further enhance model training, we also incorporated synthetic point clouds. Practical results on real point clouds, acquired from operational facilities and pre-processed as required by commercial DT solutions, indicated a performance of approximately 70% success (f1-score) in segmenting components of O&G production systems, reaching up to 85% for components such as control valves. Accordingly, a 52.4% reduction in costs and efforts associated with manually interpreting 3D point clouds is estimated. Point cloud instance segmentation has attracted much attention in the computer vision research community, and many approaches have been proposed. However, results in the literature are heavily biased toward a few benchmarking datasets, which were acquired in indoor environments like offices and rooms. Few studies have addressed the O&G domain, creating a gap in the literature for achieving more competitive results. This lack of comprehensive research limits the potential for innovation and optimization in the industry. This paper contributes by (i) assessing the performance of state-of-the-art algorithms in the domain of O&G facilities, (ii) proposing modifications to better generalize to components of O&G production systems, (iii) objectively measuring the impact of this algorithm in reducing manual effort for engineers, and (iv) advancing towards a new DT functionality that derives an asset 3D representation, that is faithful to physical reality, through the automatic interpretation of point cloud scans.

The objective of this work is to develop and apply a spectral graph theoretic approach for differentiating between (classifying) additive manufactured (AM) parts contingent on the severity of their dimensional variation from laser-scanned coordinate measurements (3D point cloud). The novelty of the approach is in invoking spectral graph Laplacian eigenvalues as an extracted feature from the laser-scanned 3D point cloud data in conjunction with various machine learning techniques. The outcome is a new method that classifies the dimensional variation of an AM part by sampling less than 5% of the 2 million 3D point cloud data acquired (per part). This is a practically important result, because it reduces the measurement burden for postprocess quality assurance in AM—parts can be laser-scanned and their dimensional variation quickly assessed on the shop floor. To realize the research objective, the procedure is as follows. Test parts are made using the fused filament fabrication (FFF) polymer AM process. The FFF process conditions are varied per a phased design of experiments plan to produce parts with distinctive dimensional variations. Subsequently, each test part is laser scanned and 3D point cloud data are acquired. To classify the dimensional variation among parts, Laplacian eigenvalues are extracted from the 3D point cloud data and used as features within different machine learning approaches. Six machine learning approaches are juxtaposed: sparse representation, k-nearest neighbors, neural network, naïve Bayes, support vector machine, and decision tree. Of these, the sparse representation technique provides the highest classification accuracy (F-score > 97%).

The study of convolutional neural networks for 3D point clouds is becoming increasingly popular, and the difficulty lies mainly in the disorder and irregularity of point clouds. At present, it is straightforward to propose a convolution operation and perform experimental validation. Although good results are achieved, the principles behind them are not explained—i.e., why this can solve the disorder and irregularity of point clouds—and it is difficult for the researchers to design a point cloud convolution network suitable for their needs. For this phenomenon, we propose a point convolution network framework based on spatial location correspondence. Following the correspondence principle can guide us in designing convolution networks adapted to our needs. We analyzed the intrinsic mathematical nature of the convolution operation, and we argue that the convolution operation remains the same when the spatial location correspondence between the convolution kernel points and the convolution range elements remains unchanged. Guided by this principle, we formulated a general point convolution framework based on spatial location correspondence, which explains how to handle a disordered point cloud. Moreover, we discuss different kinds of correspondence based on spatial location, including M-to-M, M-to-N, and M-to-1 relationships, etc., which explain how to handle the irregularity of point clouds. Finally, we give the example of a point convolution network whose convolution kernel points are generated based on the sample’s covariance matrix distribution according to our framework. Our convolution operation can be applied to various point cloud processing networks. We demonstrated the effectiveness of our framework for point cloud classification and semantic segmentation tasks, achieving competitive results with state-of-the-art networks.

Plant phenotypic analysis is of great importance to the development of agricultural engineering, and is one of the core issues in crop science and plant breeding. Since plant growth is spatio-temporal and synchronous, understanding the growth and development of individual plants can help to reveal the growth potential of the whole plot and thus improve planting methods. In recent years, the technical means to analyze the growth situation using 3D point cloud data has received extensive attention. The plant point cloud obtained by scanning plants with LiDAR has the characteristics of high resolution, high precision, etc. Periodic scanning of the same plant for spatio-temporal point cloud data sets allows monitoring of growth through subtle changes of plant organs. Organ tracking of growing plants remains challenging due to the lightward nature of growth, the potential for topological changes and the unpredictability of plant growth over time, with the possibility of new leaf growth and leaf death. This paper designs a plant organ growth tracking method based on point cloud. First of all, for growing plants, we have established a crop point cloud spatio-temporal dataset based on two publicly available point cloud datasets. The data set includes four species, tomato, tobacco, sorghum and maize, each species contains complete organ instance labels, and each organ of the same plant has a unique label, which means that the labels of the same organ of an individual at different scan dates correspond one-to-one. Second, this paper proposes a point cloud data-based plant organ growth tracking method, which uses a cost correlation matrix to automatically track growing plant organs. Finally, based on the set of quantitative evaluation metrics, our algorithm achieves a matching accuracy of 82.89% on the plant spatio-temporal dataset and good growth tracking results in the qualitative analysis.

Building Information Modeling (BIM) has gained wider acceptance in the Architecture/Engineering/Construction (A/E/C) industries in the U.S. and internationally. Capability of BIM technologies in early detection of spatial conflicts among various building systems and components concurrently designed by different design team members prior to construction has great potential in reducing construction delays, loss of productivity, and overall construction costs. Recent advancements in 3D point cloud technologies, such as 3D laser scanning, can greatly augment BIM technology applications in the A/E/C industry by providing a rapid three-dimensional capture of the built environment in great detail and accuracy. This paper discusses the current approaches of implementing 3D point cloud data in BIM and virtual design and construction (VDC) applications during various stages of a project lifecycle and the challenges associated with processing the huge amount of 3D point cloud data. Conversion from discrete 3D point cloud raster data to geometric/vector BIM data remains to be a labor-intensive process. The needs for intelligent geometric feature detection/reconstruction algorithms for automated point cloud processing and issues related to data management will be discussed. This paper also presents some innovative approaches for integrating 3D point cloud data with BIM to efficiently augment built environment design, construction, and management.

In recent years, Convolutional Neural Networks (CNNs) have been widely successful for numerous computer vision tasks using 2D color and 3D point data, such as object detection and classification, and 3D pose estimation. Despite these successes, it comes with a price --- CNNs require a vast amount of labeled training data to be successful. Many training datasets are now available, however, they are mostly limited to general objects in the public domain such as people, cars, buildings, fruit, and more. Thus, not all fields have sufficient training data. Even if an adequate number of data samples can be built into a dataset, manually collecting and labeling the training data remains a laborious task. Several approaches to reduce or eradicate the need for training data have been proposed by researchers. The two most popular approaches are training with synthetic data and training on generalized object features. Training with synthetic data refers to the use of computer graphics to generate training data. While this approach has demonstrated success, it comes with its own challenges. In contrast, training a CNN on generalized object features identifies specific local 3D features or control points on objects. Local 3D features and their descriptors have proven in the past to be universally applicable and effective for general object detection, and thus, this method allows for a generalized CNN. A generalized CNN refers to a network architecture that, after initial training, can be specialized for a new task with a minimum amount or no new training data. This dissertation investigates an encoder-decoder architecture and determines the capabilities of it to generalize the CNN architecture for object detection tasks in point cloud data. A CNN is trained to recognize 3D features and demonstrates, without retraining, its successful applicability for point cloud object detection using disparate data. Additionally, the geometric consistency of the descriptors is evaluated to obtain further insight. The results demonstrate that the proposed 3D feature descriptors, and their increased geometric consistency contribute to the increased encoder-decoder architecture performance. In summary, this dissertation contributes a state-of-the-art, generalized, feature descriptor-based CNN architecture that can be transferred to different objects without retraining. Furthermore, it provides insight explaining the increased performance of the CNN compared to the state-of-the-art.

Smart Merging Method for Hybrid Point Cloud Data using UAV and LIDAR in Earthwork Construction

Local aggressive and physically realizable adversarial attacks on 3D point cloud

Classification and recognition of objects for images is an important issue in many scientific fields such as computer vision, biometrics and medical image analysis. An important feature of many objects is shape, so shape analysis has become an important part of classification. One method of shape analysis is to estimate boundaries and analyze the shape of these boundaries to determining the characteristics of the original object. However, many literature studies on point cloud shape analysis are based on existing shapes. This paper mainly refers to Ripley’s K-function in spatial point analysis, through this judgment on spatial distribution of point cloud data to determine the existence of shape in point cloud data, through the spatial distribution of 2D point cloud data and 3D point cloud data. Judging by the randomness of the experiment, K-function has a considerable effect on judging existence of point cloud data shape through relevant experimental verification analysis.

3D point cloud data is now preferred by researchers to generate 3D models. These models can be used throughout a variety of applications including 3D building interior models. The rise of Building Information Modeling (BIM) for Architectural, Engineering, Construction (AEC) applications has given 3D interior modelling more attention recently. To generate a 3D model representing the building interior, a laser scanner is used to collect the point cloud data. However, this data often comes with noise. This is due to several factors including the surrounding objects, lighting and specifications of the laser scanner. This paper highlights on the usage of the histogram to remove the noise data. Histograms, used in statistics and probability, are regularly being used in a number of applications like image processing, where a histogram can represent the total number of pixels in an image at each intensity level. Here, histograms represent the number of points recorded at range distance intervals in various projections. As unwanted noise data has a sparser cloud density compared to the required data and is usually situated at a notable distance from the required data, noise data will have lower frequencies in the histogram. By defining the acceptable range using the average frequency, points below this range can be removed. This research has shown that these histograms have the capabilities to remove unwanted data from 3D point cloud data representing building interiors automatically. This feature will aid the process of data preprocessing in producing an ideal 3D model from the point cloud data.