Urban scene segmentation with laser-constrained CRFs

Building Information Modeling (BIM) has gained wider acceptance in the Architecture/Engineering/Construction (A/E/C) industries in the U.S. and internationally. Capability of BIM technologies in early detection of spatial conflicts among various building systems and components concurrently designed by different design team members prior to construction has great potential in reducing construction delays, loss of productivity, and overall construction costs. Recent advancements in 3D point cloud technologies, such as 3D laser scanning, can greatly augment BIM technology applications in the A/E/C industry by providing a rapid three-dimensional capture of the built environment in great detail and accuracy. This paper discusses the current approaches of implementing 3D point cloud data in BIM and virtual design and construction (VDC) applications during various stages of a project lifecycle and the challenges associated with processing the huge amount of 3D point cloud data. Conversion from discrete 3D point cloud raster data to geometric/vector BIM data remains to be a labor-intensive process. The needs for intelligent geometric feature detection/reconstruction algorithms for automated point cloud processing and issues related to data management will be discussed. This paper also presents some innovative approaches for integrating 3D point cloud data with BIM to efficiently augment built environment design, construction, and management.

Fast 3D reconstruction with semantic information in road scenes is of great requirements for autonomous navigation. It involves issues of geometry and appearance in the field of computer vision. In this work, we propose a fast 3D semantic mapping system based on the monocular vision by fusion of localization, mapping, and scene parsing. From visual sequences, it can estimate the camera pose, calculate the depth, predict the semantic segmentation, and finally realize the 3D semantic mapping. Our system consists of three modules: a parallel visual Simultaneous Localization And Mapping (SLAM) and semantic segmentation module, an incrementally semantic transfer from 2D image to 3D point cloud, and a global optimization based on Conditional Random Field (CRF). It is a heuristic approach that improves the accuracy of the 3D semantic labeling in light of the spatial consistency on each step of 3D reconstruction. In our framework, there is no need to make semantic inference on each frame of sequence, since the 3D point cloud data with semantic information is corresponding to sparse reference frames. It saves on the computational cost and allows our mapping system to perform online. We evaluate the system on road scenes, e.g., KITTI, and observe a significant speed-up in the inference stage by labeling on the 3D point cloud.

In this paper, we propose a novel approach of video segmentation into scenes based on the technique of conditional random fields (CRFs). This approach is built upon the design in which scene segmentation is transformed into a label identification problem by defining three types of shots. To implement our algorithm, three middle-level features including shot difference signal, scene transition graph and audio type are extracted to depict the label properties of each shot, and then CRFs model is employed to identify the labels sequence. The advantage of CRFs model lies in its facility in integrating context information of neighboring shots, which produces accurate results in scene segmentation. The proposed approach is verified by seven types of data covering the most major genres of TV program. Experiments on testing data set yield average 0.88 F-measure, which illustrates that the proposed method can accurately detect most scenes in different genres of programs.

Reliable feature extraction from 3D point cloud data is an important problem in many application domains, such as reverse engineering, object recognition, industrial inspection, and autonomous navigation. In this paper, a novel method is proposed for extracting the geometric features from 3D point cloud data based on discrete curves. We extract the discrete curves from 3D point cloud data and research the behaviors of chord lengths, angle variations, and principal curvatures at the geometric features in the discrete curves. Then, the corresponding similarity indicators are defined. Based on the similarity indicators, the geometric features can be extracted from the discrete curves, which are also the geometric features of 3D point cloud data. The threshold values of the similarity indicators are taken from[0,1], which characterize the relative relationship and make the threshold setting easier and more reasonable. The experimental results demonstrate that the proposed method is efficient and reliable.

This work deals with development of algorithm for physical replication of patient specific human bone and construction of corresponding implants/inserts RP models by using Reverse Engineering approach from non-invasive medical images for surgical purpose. In medical field, the volumetric data i.e. voxel and triangular facet based models are primarily used for bio-modelling and visualization, which requires huge memory space. On the other side, recent advances in Computer Aided Design (CAD) technology provides additional facilities/functions for design, prototyping and manufacturing of any object having freeform surfaces based on boundary representation techniques. This work presents a process to physical replication of 3D rapid prototyping (RP) physical models of human bone from various CAD modeling techniques developed by using 3D point cloud data which is obtained from non-invasive CT/MRI scans in DICOM 3.0 format. This point cloud data is used for construction of 3D CAD model by fitting B-spline curves through these points and then fitting surface between these curve networks by using swept blend techniques. This process also can be achieved by generating the triangular mesh directly from 3D point cloud data without developing any surface model using any commercial CAD software. The generated STL file from 3D point cloud data is used as a basic input for RP process. The Delaunay tetrahedralization approach is used to process the 3D point cloud data to obtain STL file. CT scan data of Metacarpus (human bone) is used as the case study for the generation of the 3D RP model. A 3D physical model of the human bone is generated on rapid prototyping machine and its virtual reality model is presented for visualization. The generated CAD model by different techniques is compared for the accuracy and reliability. The results of this research work are assessed for clinical reliability in replication of human bone in medical field.

The objective of this work is to develop and apply a spectral graph theoretic approach for differentiating between (classifying) additive manufactured (AM) parts contingent on the severity of their dimensional variation from laser-scanned coordinate measurements (3D point cloud). The novelty of the approach is in invoking spectral graph Laplacian eigenvalues as an extracted feature from the laser-scanned 3D point cloud data in conjunction with various machine learning techniques. The outcome is a new method that classifies the dimensional variation of an AM part by sampling less than 5% of the 2 million 3D point cloud data acquired (per part). This is a practically important result, because it reduces the measurement burden for postprocess quality assurance in AM—parts can be laser-scanned and their dimensional variation quickly assessed on the shop floor. To realize the research objective, the procedure is as follows. Test parts are made using the fused filament fabrication (FFF) polymer AM process. The FFF process conditions are varied per a phased design of experiments plan to produce parts with distinctive dimensional variations. Subsequently, each test part is laser scanned and 3D point cloud data are acquired. To classify the dimensional variation among parts, Laplacian eigenvalues are extracted from the 3D point cloud data and used as features within different machine learning approaches. Six machine learning approaches are juxtaposed: sparse representation, k-nearest neighbors, neural network, naïve Bayes, support vector machine, and decision tree. Of these, the sparse representation technique provides the highest classification accuracy (F-score > 97%).

Afterthe construction of subway main structure, in order to realize route adjustmentof alignment and gradient, it needs to survey the cross-section of subwaytunnel. Compared with conventional measuring methods, 3D laser scanning has thecharacteristics of non-contact measurement and can collect space 3D point clouddata with high density, this can improve the working efficiency for the subwaycross-section surveying. Based on the Leica Scanstation 2 scanners this paperanalyzed the 3D laser scanning point cloud data collection procedures and dataprocessing, expounded the subway cross-section surveying method based on pointcloud data; analyzed the feasibility of 3D laser scanning technology in theapplication of tunnel cross-section surveying based on the field validationdata. The results show that the cross-section measured by this method can meetthe technical requirements of route adjustment of alignment and gradient.

The boundary of a dangerous rock mass is controlled by the structures of rock masses, and the structural plane information can be obtained through 3D laser point clouds, which provides a data basis for risk assessments. This paper first introduces the principle of terrestrial 3D laser scanning, as well as the identification and interpretation methods of rock mass structural planes. Aiming at a high-steep slope on the left bank of a hydropower station, the location of a unstable rock mass is accurately identified based on 3D point cloud data, and its geometric dimensions and controllable structural plane occurrence are measured. The instability process of the rock mass is simulated using discrete elements, and the results show that the movement has experienced five stages: plane sliding, free falling, collision bouncing, rolling, continuous rebound until static. The maximum speed of the falling rock can reach 84.26 m/s, and the maximum bounce height and horizontal distance are 30m and 40m respectively, posing a great threat to the safety of structures and personnel during construction. This research can provide references to the identification and hazard risk assessment of dangerous rock masses on high-steep slopes.

Development of vehicle- recognition method on water surfaces using LiDAR data: SPD2 (spherically stratified point projection with diameter and distance)

Information of unordered point cloud is limited because of no direct topologic relation between points or triangular facets. So it will be difficult to obtain the feature points of 3D point cloud data. In this article, we use the geometry properties, such as normal, curvature and density of the points' information to detect features of the 3D point cloud data and propose a curvature and density based feature point detection method for unordered 3D point cloud data. Firstly, we define a feature parameter of 3D point cloud data, which includes the distance with its neighboring points, the sum of the normal angle between the point and neighboring points, and point cloud data curvature. Secondly, the density of data points is calculated by using Octree and is used as the features of points by a threshold of their feature parameter. The experimental results show that our new approach might detect feature points accurately for the given 3D point cloud data.

This paper presents a method with dynamic density topological structure generation for low-cost real-time vertical ladder detection from 3D point cloud data. Dynamic Density Growing Neural Gas (DD-GNG) is proposed to generate a dynamic density of the topological structure. The density of the structure and the number of nodes will be increased in the targeted object area. Feature extraction model is required to classify suspected objects for being processed in the next time process. After that, rungs of the vertical ladder is processed using an inlier-outlier method. Thus, the ladder detection model represents the ladder with a set of nodes and edges. Next, affordance detection is processed for detecting the feasible grasped location. To validate the effectiveness of the proposed method, a series of experiments are conducted on a 4-legged robot with a non-GPU board for real-time vertical ladder detection and climbing to validate the effectiveness of the proposed method. Results show that our proposed method able to detect and track the ladder structure in real-time with a much lower computational cost. The affordance of the ladder provides safety information for robot grasping.

Industrial anomaly detection by integrating RGB and 3D point cloud data has demonstrated significant potential. However, existing methods predominantly rely on 3D data, underutilizing the complementary value of RGB information. To address this, we propose a dual-branch framework that processes 2D and 3D modalities separately. In the 2D branch, we design an enhanced single-step guided diffusion model. Unlike conventional approaches that solely predict noise for reconstruction, we argue that anomalies and noise are fundamentally distinct, and explicit modeling of anomalous features is essential for effective restoration. We innovatively define the network objective as the simultaneous prediction of both noise and anomaly components. By injecting synthetic anomalies into normal samples during training, the network learns the joint distribution of "noise-anomaly". This design enables the model to more accurately identify and remove anomalies while better restoring normal structures, significantly improving reconstruction quality. To maintain efficiency, we introduce a single-step denoising mechanism based on a normalized guidance strategy, where coarser noise scales guide reconstruction at finer scales, achieving improved reconstruction accuracy while substantially accelerating inference.In the 3D branch, we build upon the 3DSR baseline and integrate Structural Similarity Index (SSIM) loss to enhance structure preservation, particularly improving detail and edge recovery, thereby increasing detection accuracy and robustness. Furthermore, we propose a dynamic learning factor strategy to optimize training convergence and stability. Finally, the outputs of both branches are fused using a One-Class Support Vector Machine (SVM) for more reliable anomaly discrimination. Our method achieves state-of-the-art performance on the MVTec 3D dataset, attaining 99.5% image-level AUROC and 99.5% pixel-level AUROC, outperforming existing mainstream approaches. The code is available at: https://github.com/cz123456-china/Diff3d-ad/.

The Bayesian approach and especially the maximum aposteriori (MAP) estimator is most widely used to solve variousproblems in signal and image processing, such as denoising anddeblurring, zooming, and reconstruction. The reason is that itprovides a coherent statistical framework to combine observed(noisy) data with prior information on the unknown signal or imagewhich is optimal in a precise statistical sense. This paperpresents an objective critical analysis of the MAP approach. Itshows that the MAP solutions substantially deviate from both thedata-acquisition model and the prior model that underly the MAPestimator. This is explained theoretically using severalanalytical properties of the MAP solutions and is illustratedusing examples and experiments. It follows that the MAP approachis not relevant in the applications where the data-observation andthe prior models are accurate. The construction of solutions(estimators) that respect simultaneously two such models remainsan open question.

Three-dimensional (3D) semantic segmentation based on point clouds has become a critical technology in the field of intelligent spatial perception and scene understanding. However, most existing methods cannot fully exploit the spatial feature of 3D point cloud data and the orderliness of two-dimensional (2D) image information, resulting in an inability to effectively improve segmentation accuracy. This study proposes a novel 2D image and 3D point cloud fusion method named shared multi-layer perceptron fusion semantic segmentation (SMFusionSeg) for semantic segmentation. The rich features of 2D image segmentation and spatial information of 3D point clouds are integrated by designing a fusion architecture that facilitates the transfer of 2D semantics to 3D semantics. Specifically, the fusion architecture concatenates the features from both 2D and 3D domains, which are then further integrated through a shared multi-layer perceptron (MLP) and optimized using an attention mechanism to enhance feature effectiveness and distinctiveness. Moreover, a knowledge distillation strategy is introduced to improve the learning capability and segmentation accuracy in parsing complex scenes. Experimental results on the SemanticKITTI dataset demonstrate that the proposed SMFusionSeg achieves a mean Intersection over Union ( mIoU ) of 65.4%, significantly outperforming traditional single-modal methods and many existing segmentation techniques, thereby effectively enhancing the precision and robustness of geographic feature recognition.

Medical imaging is becoming indispensable for patients' healthcare. Machine learning plays an essential role in the medical imaging field, including computer-aided diagnosis, image segmentation, image registration, image fusion, image-guided therapy, image annotation, and image database retrieval. With advances in medical imaging, new imaging modalities and methodologies such as cone-beam/multislice CT, 3D ultrasound imaging, tomosynthesis, diffusion-weighted magnetic resonance imaging (MRI), positron-emission tomography (PET)/CT, electrical impedance tomography, and diffuse optical tomography, new machine-learning algorithms/applications are demanded in the medical imaging field. Because of large variations and complexityit is generally difficult to derive analytic solutions or simple equations to represent objects such as lesions and anatomy in medical images. Therefore, tasks in medical imaging require “learning from examples” for accurate representation of data and prior knowledge. Because of its essential needs, machine learning in medical imaging is one of the most promising, growing fields. The main aim of this special issue is to help advance the scientific research within the broad field of machine learning in medical imaging. The special issue was planned in conjunction with the International Workshop on Machine Learning in Medical Imaging (MLMI 2010) [1], which was the first workshop on this topic, held at the 13th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI 2010) in September, 2010, in Beijing, China. This special issue is one in a series of special issues of journals on this topic [2]; it focuses on major trends and challenges in this area, and it presents work aimed at identifying new cutting-edge techniques and their use in medical imaging. The quality level of the submissions for this special issue was very high. A total of 17 papers were submitted to this issue in response to the call for papers. Based on a rigorous review process, 10 papers (59%) were accepted for publication in the special issue. The special issue starts by a review of studies on a class of machine-learning techniques, called pixel/voxel-based machine learning, in medical imaging by K. Suzuki. A series of medical imaging applications of machine-learning techniques are presented. A large variety of applications are well represented here, including organ modeling by D. Wang et al. and X. Qiao and Y.-W. Chen, brain function estimation by V. Michel et al., image reconstruction by H. Shouno et al., lesion classification by P. Wighton et al., modality classification by X.-H. Han and Y.-W. Chen, lesion segmentation by M. Zortea et al., organ segmentation by S. Alzubi et al., and visualization of molecular signals by F. Mattoli et al. Also, the issue covers various biomedical imaging modalities, including MRI by D. Wang et al., CT by X. Qiao and Y.-W. Chen and H. Shouno et al., functional MRI by V. Michel et al., dermoscopy by P. Wighton et al. and M. Zortea et al., scintigraphy by X.-H. Han and Y.-W. Chen, ultrasound imaging by X.-H. Han and Y.-W. Chen, radiography by X.-H. Han and Y.-W. Chen, MR angiography by S. Alzubi et al., and microscopy by F. Mattoli et al. as well as a variety of organs, including the kidneys by D. Wang et al., liver by X. Qiao and Y.-W. Chen, brain by V. Michel et al. and F. Mattoli et al., chest by S. Alzubi et al., skin by P. Wighton et al. and M. Zortea et al., and heart by F. Mattoli et al. Various machine-learning techniques were developed/used to solve the respective problems, including structured dictionary learning by D. Wang et al., generalized N-dimensional principal component analysis by X. Qiao et al., multiclass sparse Bayesian regression by V. Michel et al., Bayesian hyperparameter inference by H. Shouno et al., supervised learning of probabilistic models based on maximum aposteriori estimation and conditional random fields by P. Wighton et al., joint kernel equal contribution in support vector classification by X.-H. Han and Y.-W. Chen, and iterative hybrid classification by M. Zortea et al. We are grateful to all authors for their excellent contributions to this special issue and to all reviewers for their reviews and constructive suggestions. We hope that this special issue will inspire further ideas for creative research, advance the field of machine learning in medical imaging, and facilitate the translation of the research from bench to bedside. Kenji Suzuki Pingkun Yan Fei Wang Dinggang Shen