Planar patch extraction with noisy depth data

We present two robust and novel algorithms to model a 3D environment using both intensity and range data provided by an off-the-shelf stereo camera. The main issue we need to address is that the output of the stereo system is both sparse and noisy. To overcome this limitation,, we detect planar patches in the environment by region segmentation in 2D and plane extraction in 3D. The extracted planar patches are used not only to represent the workspace, but also to fill holes in range data. We also suggest a new planar patch based scan matching algorithm to register multiple views, and to incrementally augment the description of the 3D workspace in a sequence of scenes. Experimental results on real data show that planar patch segmentation and 3D scene registration for environment modeling can be robustly achieved by the proposed approaches.

Indoor scene understanding is an active research topic that has recently gained increasing attention in both computer graphics and computer vision community. This paper presents a novel automatic point cloud instance abstraction and segmentation of indoor scenes using learned pairwise planar patch relations. Planar patches have sufficient representative power to abstract the semantics in mostly man-made 3D indoor scenes. To exploit the planar patch relationships, a new indoor point clouds dataset is automatically derived from the public dataset ScanNet, which can be adopted for training a pairwise patch relation classifier. Given the pre-processed indoor scene point clouds, a set of planar patches are generated to provide robust and compact representation of indoor objects and parts. For each extracted planar patch, a feature descriptor in view of its geometric properties and appearance is calculated. Owing to the proposed feature descriptors, the relations of pairwise planar patch are measured and can be fed into the pre-trained patch relation classifier. To yield the instance segmentation of indoor scenes, a robust graph-based algorithm is adopted to group these planar patches which incorporate the learned patch relations. Experimental results demonstrate that our proposed approach can produce instance semantic segmentation of various cluttered indoor scenes robustly and efficiently.

As an autonomous robot navigates in an unknown environment, the perception subsystem must be able to perform the incremental modeling of this environment as well as robot self-location. In this paper, the authors have chosen to combine a polyhedral representation of the world with its digital elevation model, in order to deal with autonomous navigation in semi-structured environments. The authors describe algorithms needed to build snapshot models from noisy and sparse range data, to perform 3D data fusion between these snapshot models, in order to incrementally build a reliable 3D model from which a path planner could generate safe trajectories. The authors present a fully implemented modeling strategy based on range data processing, and describe the different modeling services required in order to perform the indoor scene navigation of the robot HILARE-2.

For any modern 3D vision guided system, it is imperative to have complete range images for 3D model reconstruction. In practice, depth images obtained from a standard stereo camera can be error-prone with missing depth pixels. This paper proposes a method to augment the range data obtained from stereoscopy with model based image segmentation using planar patches. The method integrates both intensity and range data obtained from a standard stereo system. First, edges are extracted and linked to different segments from the intensity image with the embedded confidence edge detection technique and a general edge-linking algorithm respectively. Since edges are where disparity happens, most straight-line edges segmented from linked edges using a line-curvature extraction algorithm will have valid depth data. The planar patches are then defined by the straight-lines edge. With the knowledge of the planar structure, each depth missing pixels in the region is then determined by various 3-D line equations that pass through the pixel in the world coordinate system. Lastly, the range data in the world coordinate system is converted back into the image coordinate system using a pixel-to-pixel project algorithm. Result demonstrates the accuracy of method for filling up the missing depth in a region.

<p>LiDAR (Light Detection and Ranging) is a technology that provides three-dimensional point cloud data with high spatial accuracy. It is increasingly used in various applications and disciplines, including engineering. Along with other capabilities, LiDAR systems are able to record reflected backscattered energy as intensity data and measured distances to targets as range data. As the laser signal wavelength that operates in LiDAR sensors is typically in the near-infrared (NIR) spectrum, high spectral reflectance separability can be observed and a number of different materials distinguished. These capabilities have motivated researchers to study the applicability of using LiDAR range and intensity data for pavement crack extraction.</p> <p>The main goal of the present research is to examine the potential use of LiDAR range and intensity data for automated pavement crack detection. The study explores the following: a) crack detection using the two image classification logics of pixel-based and object-based image classification; b) crack detection using the Maximum Likelihood Classifier (MLC), the Random Trees Classifier (RTC), and the Support Vector Machine (SVM) classifier; c) crack detection by conducting image classification of four scenarios of multi-layer images generated from LiDAR data; and d) crack detection using LiDAR data with different point spacing.</p> <p>The experimental results indicate that the use of LiDAR data independently is effective for pavement crack detection. In addition, the results show that object-based image classification logic optimizes classification accuracy and crack detection results, and that employing SVM enhances the classification process and improves crack detection accuracy. It is also observed that using range data separately produces high accuracy for the crack detection results while the accuracy is slightly reduced when intensity data are combined with range data compared with using range data only. Furthermore, in the classification process, adding slope and aspect layers to range and intensity data decreases the accuracy of crack detection results, while increasing LiDAR data point spacing reduces crack extraction accuracy.</p>

On deep learning techniques to boost monocular depth estimation for autonomous navigation

In this paper we address the problem of dense depth map estimation from sparse noisy range data to reconstruct large heterogeneous outdoor scenes. We propose a surface inpainting solution through energy minimisation with an adaptive selection of surface regularisers among a set of well known convex and non-convex regularisers. In fact, the selection of norm is pivotal with respect to the intrinsic surface characteristics. Our goal is to show how dense interpolation of sparse range data can be leveraged of more exotic and non-convex regularisers such as the log and logTGV [1] which can better capture the scene geometry. In contrast to state of the art solutions, we do not restrict ourselves to this set of norms, instead we search for the most apt norm for each semantically segmented part of the scene. Our energy model selection use Bayesian optimisation to learn the best choice of free parameters. This results in an adaptive model selection and the generalisation of well studied regularisation norms. We conclude with a detailed experimental analysis of our approach using a basis of four norms over a set of challenging outdoor scenes.

Perception is the integration of various sensor outputs into a unified vision of the world. A major part of research in machine vision has been limited to the use of individual sensors for building machine vision systems. The use of multisensor data is advocated for object recognition systems, and an algorithm for fusion of intensity and range data for segmentation is proposed. The algorithm consists of two steps. First, the initial seed segmentation is achieved by using the most dominating sensor at a given time. For this purpose the distributions of the intensity and range data are considered, and the image is segmented recursively by using the most significant peak in both histograms. Second, the initial segmentation is refined by using region merging; the regions are merged if the combined strengths of range and intensity boundaries are low. The experimental results for synthetic and real scenes are presented.

The success of deep learning models in fluid dynamics applications will depend on their ability to handle sparse and noisy data accurately. This paper concerns the development of a deep learning model for reconstructing turbulent flow images from low-resolution counterparts encompassing noise. The flow is incompressible through a symmetric, sudden expansion featuring bifurcation, instabilities, and turbulence. The deep learning model is based on convolutional neural networks, in a high-performance, lightweight architecture. The training is performed by finding correlations between high- and low-resolution two-dimensional images. The study also investigates how to remove noise from flow images after training the model with high-resolution and noisy images. In such flow images, the turbulent velocity field is represented by significant color variations. The model's peak signal-to-noise ratio is 45, one of the largest achieved for such problems. Fine-grained resolution can be achieved using sparse data at a fraction of the time required by large-eddy and direct numerical simulation methods. Considering its accuracy and lightweight architecture, the proposed model provides an alternative when repetitive experiments are complex and only a small amount of noisy data is available.

Industrial robots are used especially in sectors where the human body is in danger or is working in extreme conditions.One of these sectors is the forges sector where the manipulating objects are hot, and the vibrations and noise are big.The problem of manipulation the hot parts implies the choosing of the right gripper mechanism in order to obtain the best cooling of it and of the robot itself.The cooling of gripper and robot is realized in the movement phase when no hot payload is present.So, the trajectory of movement is important and also the optimum position of robot with respect to the application positions.The proposed paper is dealing with the subject of moving the base of robot with respect to the application points so that an objective function representing the accumulated heat during a cycle to be minim.

<title>Abstract</title> Conventional optimization algorithms for AR markerless registration often converge to local minima under noisy depth data. We propose a quantum-inspired optimization approach that leverages a variational quantum annealing algorithm simulated on classical hardware to enhance global convergence in point cloud alignment. The method introduces probabilistic tunneling to escape poor local optima, integrated with adaptive ICP refinement. In 500 phantom trials, our method reduced registration error from 2.4 mm (ICP) and 1.7 mm (genetic algorithm) to 1.0 mm, while requiring 35% fewer iterations. Average runtime was 41 ms per frame, sustaining real-time AR visualization at 24 fps. This demonstrates the potential of quantum-inspired algorithms in advancing surgical AR registration accuracy and efficiency.

Humans can interact with several kinds of machine (motor vehicle, robots, among others) in different ways. One way is through his/her head pose. In this work, we propose a head pose estimation framework that combines 2D and 3D cues using the concept of key frames (KFs). KFs are a set of frames learned automatically offline that consist the following: 2D features, encoded through Speeded Up Robust Feature (SURF) descriptors; 3D information, captured by Fast Point Feature Histogram (FPFH) descriptors; and target’s head orientation (pose) in real-world coordinates, which is represented through a 3D facial model. Then, the KF information is re-enforced through a global optimization process that minimizes error in a way similar to bundle adjustment. The KF allows to formulate, in an online process, a hypothesis of the head pose in new images that is then refined through an optimization process, performed by the iterative closest point (ICP) algorithm. This KF-based framework can handle partial occlusions and extreme rotations even with noisy depth data, improving the accuracy of pose estimation and detection rate. We evaluate the proposal using two public benchmarks in the state of the art: (1) BIWI Kinect Head Pose Database and (2) ICT 3D HeadPose Database. In addition, we evaluate this framework with a small but challenging dataset of our own authorship where the targets perform more complex behaviors than those in the aforementioned public datasets. We show how our approach outperforms relevant state-of-the-art proposals on all these datasets.

Human body reshaping and its application using multiple RGB-D sensors

The theory and use of a laser sensor designed to provide arrays of registered intensity and range data for three-dimensional scene analysis are described in this paper. Equations are derived to relate the mean and variance of both the intensity and the range data to the physical properties of the target and the parameters of the sensor. Experimental results that verify these equations are given. Adaptive data smoothing and normalization procedures are presented. These procedures reduce photon noise and correct the intensity data for inverse square-law effects. The resulting intensity data give the product of the diffuse surface reflectance and the cosine of the angle of incidence for each picture element in the scene. Simple procedures are given for finding outlines of occluding objects, obtaining normal views of obliquely sensed planar surfaces, and extracting the images of horizontal surface regions from a scene. These procedures demonstrate the advantages of having registered reflectance and range data for scene analysis.

The authors present a parallel interpretation tree search algorithm for object recognition using sparse range or tactile data on the Intel iPSC/2 hypercube multicomputer. The objects are typically those which can be approximated as a piecewise combination of polyhedra and which a robot would encounter in an industrial scene during the process of automated inspection or automated assembly. Three strategies for mapping the interpretation tree search process on the hypercube are considered. These are breadth-first mapping, depth-first mapping and depth-first mapping with load sharing. The algorithm has been experimentally verified on synthetic tactile data from two-dimensional scenes. It has shown that the requirement for uniform load sharing leads to increased inter-processor communication. >