Range Image Segmentation for Modeling and Object Detection in Urban Scenes

An adaptive range image coding algorithm for the geometry compression of large-scale 3D point clouds (LS3DPCs) is proposed in this work. A terrestrial laser scanner generates an LS3DPC by measuring the radial distances of objects in a real world scene, which can be mapped into a range image. In general, the range image exhibits different characteristics from an ordinary luminance or color image, and thus the conventional image coding techniques are not suitable for the range image coding. We propose a hybrid range image coding algorithm, which predicts the radial distance of each pixel using previously encoded neighbors adaptively in one of three coordinate domains: range image domain, height image domain, and 3D domain. We first partition an input range image into blocks of various sizes. For each block, we apply multiple prediction modes in the three domains and compute their rate-distortion costs. Then, we perform the prediction of all pixels using the optimal mode and encode the resulting prediction residuals. Experimental results show that the proposed algorithm provides significantly better compression performance on various range images than the conventional image or video coding techniques

In this paper we present a novel algorithm for very fast segmentation of range images into both planar and curved surface patches. In contrast to other known segmentation methods our approach makes use of high-level features (curve segments) as segmentation primitives instead of individual pixels. This way the amount of data can be significantly reduced and a very fast segmentation algorithm is obtained. The proposed algorithm has been tested on a large number of real range images and demonstrated good results. With an optimized implementation our method has the potential to operate in quasi real-time (a few range images per second).

Computer vision researchers have been frustrated in their attempts to automatically derive depth information from conventional two-dimensional intensity images. Research on shape from texture, shape from shading, and shape from focus is still in a laboratory stage and had not seen much use in commercial machine vision systems. A range image or a depth map contains explicit information about the distance from the sensor to the object surfaces within the field of view in the scene. Information about surface geometry which is important for, say, three-dimensional object recognition is more easily extracted from 2 1/2 D range images than from 2D intensity images. As a result, both active sensors such as laser range finders and passive techniques such as multi-camera stereo vision are being increasingly utilized by vision researchers to solve a variety of problems. This book contains chapters written by distinguished computer vision researchers covering the following areas: Overview of 3D Vision Range Sensing Geometric Processing Object Recognition Navigation Inspection Multisensor Fusion A workshop report, written by the editors, also appears in the book. It summarizes the state of the art and proposes future research directions in range image sensing, processing, interpretation, and applications. The book also contains an extensive, up-to-date bibliography on the above topics. This book provides a unique perspective on the problem of three-dimensional sensing and processing; it is the only comprehensive collection of papers devoted to range images. Both academic researchers interested in research issues in 3D vision and industrial engineers in search of solutions to particular problems will find this a useful reference book.

Urban areas are rapidly changing all over the world and therefore provoke the necessity to update urban maps frequently. Remote sensing has been used for many years to monitor these changes. The urban scene is characterized by a very high complexity, containing objects formed from different types of man-made materials as well as natural vegetation. Hyperspectral sensors provide the capability to map the surface materials present in the scene using their spectra and therefore to identify the main object classes in the scene in a relatively easy manner. However ambiguities persist where different types of objects are constructed of the same material. This is for instance the case for roads and roof covers. Although higher-level image processing (e.g. spatial reasoning) might be able to relief some of these constraints, this task is far from straight forward. In the current paper the authors fused information gathered using a hyperspectral sensor with that of high-resolution polarimetric SAR data. SAR data give information about the type of scattering backscatter from an object in the scene, its geometry and its dielectric properties. Therefore, the information obtained using the SAR processing is complementary to that obtained using hyperspectral data. This research was applied on a dataset consisting of hyperspectral data from the HyMAP sensor (126 channels in VIS-SWIR) and E-SAR data which consists of fullpolarimetric L-band and dual-polarisation (HH and VV) X-band data. Two supervised classifications are used; 'Logistic Regression' (LR) which applied to the SAR and the PolSAR data and a 'Matched Filter' which is applied to the hyperspectral data. The results of the classification are fused in order to improve the mapping of the main classes in the scene and were compared to a ground truth map that was constructed by combining a digital topographic map and a vectorized cadastral map of the research area. An adequate change detection of man-made objects in urban scenes was obtained by the fusion of features derived from SAR, PolSAR and hyperspectral data.

A means has been developed of deriving range images from sequences of conventional images taken by a single passive camera (forward looking or side looking) on-board a vehicle as it moves continuously relative to the scene. These range images can provide the 3D situational awareness needed for autonomous operation of the vehicle. The first image in a sequence is registered in turn with each subsequent image in the sequence using a new algorithm for the registration of different views of a three-dimensional (3D) scene. A range image is derived from the registration. This range sensing method has been developed and demonstrated using a sequence of photographic images of a 3D model of an urban canyon scene, and is now being implemented in hardware.

This paper considers the segmentation of range image measurements into surface patches which are either plane or curved and which are described formally by a function. After a formal description of the segmentation, we present and compare three methods suited for plane and curved patch segmentation and show the results of experiments conducted for testing their practical behaviour. The two first methods use the classical approach of region growing whereas the third method is based on a relaxation process. This original and last method exhibits simplicity and low computational complexity. Thanks to its parallel nature, it can be considered as a good candidate for range image segmentation in real-time applications.

A technique to generate a photorealistic three-dimensional (3D) image and color-textured model using a dedicated optical digitizer is presented. The proposed technique is started with the range and texture image acquisition from different viewpoints, followed by the registration and integration of multiple range images to get a complete and nonredundant point cloud that represents a real-life object. The accuracy of the range image and the precision of correspondence between the range image and texture image are guaranteed by sensor system calibration. Based on the point cloud, a geometric model is established by considering the connectivity of adjacent range image points. In order to enhance the photorealistic effect, we suggest a texture blending technique that utilizes a composite-weight strategy to blend the texture images within the overlapped region. This technique allows more efficient removal of the artifacts existing in the registered texture image, leading to a 3D image with photorealistic quality and color-texture modeling. Experimental results are also presented to testify to the validity of the proposed method.

Multi-view range image registration is a significant and challenging problem for 3D modeling. This paper presents a reference shape based multi-view range image registration algorithm. First, a set of Rotational Projection Statistics (RoPS) features are extracted from the input range images. Next, the reference shape is initialized by selecting a range image from the input. The reference shape is then iteratively updated by registering itself with the remaining range images. The registration between the reference shape and any range image is completed by RoPS feature matching. Finally, all input range images are registered according to their corresponding reference shapes. A number of experiments were performed to test the performance of our algorithm. The experimental results show that the reference shape based algorithm can perform multi-view registration on a mixed set of unordered range images corresponding to several different objects. It is also very accurate and efficient. It outperformed the state-of-the-arts including the spanning tree based and connected graph based algorithms.

Based on the spherical harmonics representation of image formation, we derive a new photometric metric for evaluating the correctness of a given rigid transformation aligning two overlapping range images captured under unknown, distant, and general illumination. We estimate the surrounding illumination and albedo values of points of the two range images from the point correspondences induced by the input transformation. We then synthesize the color of both range images using albedo values transferred using the point correspondences to compute the photometric reprojection error. This way allows us to accurately register two range images by finding the transformation that minimizes the photometric reprojection error. We also propose a practical method using the proposed photometric metric to register pairs of range images devoid of salient geometric features, captured under unknown lighting. Our method uses a hypothesize-and-test strategy to search for the transformation that minimizes our photometric metric. Transformation candidates are efficiently generated by employing the spherical representation of each range image. Experimental results using both synthetic and real data demonstrate the usefulness of the proposed metric.

Time-of-flight range imaging cameras measure distance to objects in their field of view, but are prone to error when objects move. At least three raw frames are required to obtain one range image, and the standard method is to read out raw frames into separate sets and process to find one range image per set. Motion during the acquisition of a set causes error in the corresponding range image. In this paper, the problem of motion is addressed by regarding the raw data from each pixel as a noisy time series, and using the Kalman filter to efficiently perform time-series analysis. The proposed method adapts to the effects of transverse motion, measuring a sharp range image at each raw frame. The error in the proposed method is less than the traditional approach in 80% of tests, with no detected increase in the STD due to noise. In the qualitative experimental results, the visible blur is reduced.

Despite the ubiquitous use of range images in various computer vision applications, little has been investigated about the size variation of the local geometric structures captured in the range images. In this paper, we show that, through canonical geometric scale-space analysis, this geometric scale-variability embedded in a range image can be exploited as a rich source of discriminative information regarding the captured geometry. We extend previous work on geometric scale-space analysis of 3D models to analyze the scale-variability of a range image and to detect scale-dependent 3D features – geometric features with their inherent scales. We derive novel local 3D shape descriptors that encode the local shape information within the inherent support region of each feature. We show that the resulting set of scale-dependent local shape descriptors can be used in an efficient hierarchical registration algorithm for aligning range images with the same global scale. We also show that local 3D shape descriptors invariant to the scale variation can be derived and used to align range images with significantly different global scales. Finally, we demonstrate that the scale-dependent/invariant local 3D shape descriptors can even be used to fully automatically register multiple sets of range images with varying global scales corresponding to multiple objects.KeywordsGeodesic DistanceShape DescriptorRange ImageRegistration AlgorithmAutomatic RegistrationThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

An on-line 3D registration system is introduced to reconstruct a 3D model using a hand-held stereo camera. Multi-view range images, which are obtained from the hand-held stereo camera, are registered into a reference coordinate system simultaneously. For the coarse registration of a range image, we use the transformation of its previous frame and the centroid of foreground range images. After refining the coarse registration using a geometric feature matching technique, we use a modified KLT (Kanade-Lucas-Tomasi) tracker to match photometric features to enhance registration accuracy. We modify the KLT tracker to facilitate searching of correspondence from the results of the geometric registration. If a range image fails to register, we adjust the orientation of the camera while viewing a graphical user interaction system. After enough range images are registered, they are integrated into a 3D model in offline. Experimental results show that the proposed method can be used to reconstruct 3D models fast and accurately.

This paper presents two novel algorithms for estimating the (local and global) motion in a series of range images based on a polynomial expansion. The use of polynomial expansion has been quite successful in estimating optical flow in 2D imagery, but has not been used extensively in 3D or range imagery. In both methods, each range image is approximated by applying a high-order polynomial expansion to local neighborhoods within the range image. In the local motion algorithm, these approximations are then used to derive the translation or displacement estimation within the local neighborhoods from frame to frame within the series of range images (also known as range image flow). An iterative method for computing the local translations is presented. In the global motion algorithm, a global motion model framework is utilized to compute a global motion estimation based on the polynomial expansion of the range images. We evaluate the algorithms on several real-world range image sequences with promising results.

We presentPiCam(Pelican Imaging Camera-Array), an ultra-thin high performance monolithic camera array, that captures light fields and synthesizes high resolution images along with a range image (scene depth) through integrated parallax detection and superresolution. The camera is passive, supporting both stills and video, low light capable, and small enough to be included in the next generation of mobile devices including smartphones. Prior works [Rander et al. 1997; Yang et al. 2002; Zhang and Chen 2004; Tanida et al. 2001; Tanida et al. 2003; Duparré et al. 2004] in camera arrays have explored multiple facets of light field capture - from viewpoint synthesis, synthetic refocus, computing range images, high speed video, and micro-optical aspects of system miniaturization. However, none of these have addressed the modifications needed to achieve the strict form factor and image quality required to make array cameras practical for mobile devices. In our approach, we customize many aspects of the camera array including lenses, pixels, sensors, and software algorithms to achieve imaging performance and form factor comparable to existing mobile phone cameras.Our contributions to the post-processing of images from camera arrays include a cost function for parallax detection that integrates across multiple color channels, and a regularized image restoration (superresolution) process that takes into account all the system degradations and adapts to a range of practical imaging conditions. The registration uncertainty from the parallax detection process is integrated into a Maximum-a-Posteriori formulation that synthesizes an estimate of the high resolution image and scene depth. We conclude with some examples of our array capabilities such as postcapture (still) refocus, video refocus, view synthesis to demonstrate motion parallax, 3D range images, and briefly address future work.

A new method for stamped character recognition is proposed. Conventional techniques have used a camera to observe the characters and get the image as a gray-level. However, it has been difficult to get an exact image, because gray-level noise occurs at some distortion around the characters, which makes a shadow on the characters, and also at these parts of stamped surface which are dirty or painted. The approach presented can solve these problems by using a range finder system as the observation system. The method observes the stamped characters as range image. The authors compare the characteristics of a range image to the characteristics of a gray-level image, especially about the noise, and show a noise reduction method for range images. Also, the performance is verified by an experiment. >