Use of Local Surface Curvature Estimation for Adaptive Vision System Based on Active Light Projection

We present an approach to integrate a preprocessing step of the region of interest (ROI) localization into 3-D scanners (laser or stereoscopic). The definite objective is to make the 3-D scanner intelligent enough to localize rapidly in the scene, during the preprocessing phase, the regions with high surface curvature, so that precise scanning will be done only in these regions instead of in the whole scene. In this way, the scanning time can be largely reduced, and the results contain only pertinent data. To test its feasibility and efficiency, we simulated the preprocessing process under an active stereoscopic system composed of two cameras and a video projector. The ROI localization is done in an iterative way. First, the video projector projects a regular point pattern in the scene, and then the pattern is modified iteratively according to the local surface curvature of each reconstructed 3-D point. Finally, the last pattern is used to determine the ROI. Our experiments showed that with this approach, the system is capable to localize all types of objects, including small objects with small depth.

3D map reconstruction plays a critical role in fields such as autonomous driving, urban planning, and virtual reality. Using 2D images from real-world settings, multi-view image-based 3D map reconstruction offers a practical solution across these sectors. Specifically, generating 3D maps from realistic drone-view images has broad applicability. However, direct reconstruction from drone-view images is challenging due to extreme viewing angles and unstable image conditions like motion blur or varying light. Additionally, simultaneously generating and utilizing 3D maps from drone-captured images presents significant challenges, including fragmented sub-tasks such as 3D point cloud reconstruction, surface reconstruction, and shape fitting. To address these challenges, we introduce the 3D Scene Abstraction, which seamlessly integrates image matching, 3D reconstruction, 3D semantic segmentation, clustering, and shape fitting to produce 3D maps from RGB images. Our approach enhances the robustness of 3D map generation by merging 2D semantic information with 3D point clouds, ensuring reliable results under diverse drone-captured image conditions. For efficient data representation, we employ multiple primitives based on prior knowledge of various building shapes, improving both map formation performance and computational efficiency. Our approach not only enhances the effectiveness of 3D map reconstruction but also reduces the amount of data required to represent the scene, thereby improving overall efficiency and practical usability. These contributions address practical challenges in the field and lay the foundation for future advancements in 3D map reconstruction technologies.

In order to reconstruct the panoramic model and extract plant community structure characteristics rapidly and accurately for the highway central green belt, a method for rapidly acquiring images of the targets highway central green belt using quad rotor unmanned aerial vehicle (UAV) and digital image processing techniques is explored. It has the advantages of real-time, high flexibility and low cost to obtain the plant information of the green belt in the middle of the road based on the low-altitude unmanned aerial vehicle. The results show that the images acquired with a lower flight height (30 m and 50 m) can be used to produce rather high quality Ortho-mosaic, 3D point cloud reconstruction and ideal digital surface model. The density of 3D reconstruction point cloud for the target sections is even, and the single-pixel resolution of the ortho-mosaic image can reach 0.67 cm and 1.0 cm, respectively. Thus, it can be seen that the UAV platform can obtain high-precision panoramic reconstruction of monitoring areas with different road types.

Myosin II Controls Cellular Branching Morphogenesis and Migration in 3D by Minimizing Plasma Membrane Curvature

3D human face model reconstruction is essential to the generation of facial animations that is widely used in the field of virtual reality (VR). The main issues of 3D facial model reconstruction based on images by vision technologies are in twofold: one is to select and match the corresponding features of face from two images with minimal interaction and the other is to generate the realistic-looking human face model. In this paper, a new algorithm for realistic-looking face reconstruction is presented based on stereo vision. Firstly, a pattern is printed and attached to a planar surface for camera calibration, and corners generation and corners matching between two images are performed by integrating modified image pyramid Lucas-Kanade (PLK) algorithm and local adjustment algorithm, and then 3D coordinates of corners are obtained by 3D reconstruction. Individual face model is generated by the deformation of general 3D model and interpolation of the features. Finally, realistic-looking human face model is obtained after texture mapping and eyes modeling. In addition, some application examples in the field of VR are given. Experimental result shows that the proposed algorithm is robust and the 3D model is photo-realistic.

This paper improves the authors' conventional method for reconstructing the 3D structure of moving and still objects that are tracked in the video and/or depth image sequences acquired by moving cameras and/or range finder. The authors proposed a Temporal Modified-RANSAC based method [1] that (1) can discriminate each moving object from the still background in color image and depth image sequences acquired by moving stereo cameras or moving range finder, (2) can compute the stereo cameras' egomotion, (3) can compute the motion of each moving object, and (4) can reconstruct the 3D structure of each moving object and the background. However, the TMR based method has the following two problems concerning the 3D reconstruction: lack of accuracy of segmenting into each object's region and sparse 3D reconstructed points in each object's region. To solve these problems of our conventional method, this paper proposes a new 3D segmentation method that utilizes Graph-cut, which is frequently used for segmentation tasks. First, the proposed method tracks feature points in the color and depth image sequences so that 3D optical flows of the feature points in every N frames are obtained. Then, TMR classifies all the obtained 3D optical flows into regions (3D flow set) for the background and each moving object; simultaneously, the rotation matrix and the translation vector for each 3D flow set are computed. Next, Graph-Cut using the energy function that consists of color probability, structure probability and a-priori probability is performed so that pixels in each frame are segmented into object regions and the background region. Finally, 3D point clouds are obtained from the segmentation result image and depth image, and then the point clouds are merged using the rotation and translation from the N-th frame prior to the current frame so that 3D models for the background and each moving object are constructed with dense 3D point data.

Image-based 3D shape reconstruction is essential for 3D modeling in recognition, virtual reality, generation of video games, 3D animation, and in the creation of 3D virtual fitting rooms - the sought-after application in this dissertation. In this work, we propose a novel reconstruction method that uses a generic 3D model that roughly controls the overall reconstruction shape which in turn, reduces the number of 2D images needed to just two (frontal and profile) for obtaining a 3D point model. The generic model consists of a massive set of 3D points that lack saliency, topological meaning, and relational interconnections. This is also true about points in the two canonical images. In this work, we propose finding a small set of interconnected ordered intrinsic salient points (control points) residing on the silhouette of the projections of the generic model onto the two image spaces, as well as finding those for the canonical images. Points on the generic model and the images are automatically given saliency with order and interconnections through loop-subdivision operating on those control points and their equivalent points on the canonical images. The transformation of the generic model into the specific object is done through morphing the points on the generic model to follow and be consistent with their equivalent points on the canonical images. The control points on the canonical images are automatically obtained using an active shape model (ASM) that modifies an average trained control point set attained using an offline training, to account for class shape variability and boundary regularization guaranteeing the control points to reside on the silhouette (boundary) of the individual. The reconstructed 3D model is furtherly refined through smoothing the reconstructed surface, while fixing local holes using a principle component analysis (PCA) decomposition. This reconstruction method is convenient, simple, efficient, and results in an average geometric error of less than 0.5% over 700 diverse body models from the CAESAR dataset. We then implement the contour-based articulated model to the personalized 3D model for pose recovery. The evolution of the model through the articulation process is captured in a video clip. 3D transformations are found and applied to the 3D model on its parts so that the projections of the 3D model 'match' those observed in the video sequence at corresponding frames. This is done by minimizing the error between the frontal projection body region points and the target points from the image for each independent moving part, with sub-resolution recovery errors. Once the personalized 3D model is obtained and the articulated model is implemented, the algorithm simulates how garments virtually appear on the reconstructed person under different poses. Different types of clothes are classified as tight or loose, where for the first type of garments, the garment is mapped onto the person's body, whereas for the second type, the mapping adjusts to accommodate the loose nature of the garment, and its fitting on the body of the reconstructed person model. This creates a virtual in-home fitting room for garment fitting, which is timely and is a needed application that would go hand in hand with online clothes shopping, allowing the trying of garments at home with changing poses before ordering them, and can be updated as often as necessary to accommodate changes in physique, weight, aging, etc. This is also done with minimum requirements from the user aside from taking two images and a video of pose articulations using their cam or smart phone, and stands as a useful, needed, and attractive App.

Proceedings of the 2011 joint ACM workshop on Human gesture and behavior understanding

The interaction of atomic hydrogen and low-energy hydrogen ions with ${\mathrm{sp}}^{2}$-bonded carbon is investigated on the surfaces of ${\mathrm{C}}_{60}$ multilayer films, single-walled carbon nanotubes, and graphite (0001). These three materials have been chosen to represent ${\mathrm{sp}}^{2}$-bonded carbon networks with different local curvatures and closed surfaces (i.e. no dangling bonds). Chemisorption of hydrogen on these surfaces reduces emission from photoemission features associated with the \ensuremath{\pi} electrons and leads to a lowering of the work function up to 1.3 eV. It is found that the energy barrier for hydrogen adsorption decreases with increasing local curvature of the carbon surface. Whereas in the case of ${\mathrm{C}}_{60}$ and single-walled carbon nanotubes, hydrogen adsorption can be achieved by exposure to atomic hydrogen, the hydrogen adsorption on graphite (0001) requires ${H}^{+}$ ions of low kinetic energy (\ensuremath{\sim}1 eV). On all three materials, the adsorption energy barrier is found to increase with coverage. Accordingly, hydrogen chemisorption saturates at coverages that depend on the local curvature of the sample and the form of hydrogen (i.e., atomic or ionic) used for the treatment.

This paper presents a computationally efficient pipeline to achieve 3D point cloud reconstruction from video sequences. This pipeline involves a key frame selection step to improve the computational efficiency by generating reliable depth information from pair-wise frames. An outlier removal step is then applied in order to further improve the computational efficiency. The reconstruction is achieved based on a new absolute camera pose recovery approach in a computationally efficient manner. This pipeline is devised for both sparse and dense 3D reconstruction. The results obtained from video sequences exhibit higher computational efficiency and lower re-projection errors of the introduced pipeline compared to the existing pipelines.

In this research, we slightly modify the conventional pipeline such that it (1) sets a minimum triangulation angle of 3°, (2) reduces (and possibly minimizes) overall re-projection error by simultaneously optimizing all camera poses and 3D points in the bundle adjustment step, and (3) uses a tiling buffer size of 1024 × 1024, to generate, from a set of 2D images, a detailed 3D models of complex objects. We systematically showcase the versatility of this improved photogrammetric pipeline by applying it to three datasets - a proprietary dataset (money plant) and two publicly available datasets (a statue and an old computer). To the best of our knowledge, while previous research has employed photogrammetry techniques for 3D model generation, none has systematically defined the 3d reconstruction pipeline, with a focus on underlying mathematical concepts, Additionally, there has been a lack of evaluation of how the quality of input 2D images impacts quality of the resulting 3D models. Our study evaluates the entropy and image quality (using BRISQUE scores) of the 3D model generated in relation to the image quality of the set of input 2D images, providing insights into the reliability of the proposed photogrammetric approach. Despite utilizing a low-quality image dataset as input, we demonstrated that we achieved a high-quality 3D model. Furthermore, we observed less differences in image quality when comparing the 3D model with the original 2D images. Overall, our research contributes to both theoretical understanding and practical application of photogrammetry for 3D reconstruction as experimented on three distinct image datasets. As a future research scope, we provide suggestions for possible parallelization of some of the steps of the 3D reconstruction pipeline.

A unified differential geometric framework for estimation of local surface shape and orientation from projective distortion is proposed, based on a differential version of the stationarity assumption introduced by Malik and Rosenholtz. This framework allows the information content of the gradient of any descriptor defined in a local coordinate frame to be characterized in a very compact form. The analysis encompasses both full affine descriptors and the classical texture gradients. For estimation of local surface orientation and curvature from uncertain observations of affine distortion, the proposed framework allows the dimensionality of the search space to be reduced from five to one. >

Although coherence scanning interferometry (CSI) is capable of measuring surface topography with sub-nanometre precision, it is well known that the performance of measuring instruments depends strongly on the local tilt and curvature of the sample surface. Based on 3D linear systems theory, however, a recent analysis of fringe generation in CSI provides a method to characterize the performance of surface measuring instruments and offers considerable insight into the origins of these errors. Furthermore, from the measurement of a precision sphere, a process to calibrate and partially correct instruments has been proposed. This paper presents, for the first time, a critical look at the calibration and correction process. Computational techniques are used to investigate the effects of radius error and measurement noise introduced during the calibration process for the measurement of spherical and sinusoidal profiles. Care is taken to illustrate the residual tilt and curvature dependent errors in a manner that will allow users to estimate measurement uncertainty. It is shown that by calibrating the instrument correctly and using appropriate methods to extract phase from the resulting fringes (such as frequency domain analysis), CSI is capable of measuring the topography of surfaces with varying tilt with sub-nanometre accuracy.

Underwater 3D object reconstruction is one of the most essential and fundamental tasks in ocean investigations. In this paper, we try to capture the inherent geometrical variation of 3D objects at multiple visual angels with the underwater vehicles and develop a novel underwater 3D object reconstruction model for the continuous video stream. Image enhancement will be first taken into consideration by guided image filtering to acquire clearer images for object tracking which can be completed by the particle filter. And then, we use scale invariant feature transform (SIFT) and random sample consensus (RANSAC) to detect and correspond the features of objects we have tracked. From these correspondences, the 3D point cloud reconstruction will be finally developed with the help of structure from motion (SFM) and patches-based multi-view stereo (PMVS). It has been shown in the simulation experiments that the developed scheme of this paper achieves consistent performance improvements over 3D object reconstruction for underwater video stream in multiple views.

We explore the automatic adjustment of an artificial light source intensity for the purposes of image-based feature extraction and recognition. Two histogram-based criteria are proposed to achieve this adjustment: a two-class separation measure for 2D features and a Gaussian distribution measure for 2.5D features. To this end, the light source intensity is varied within a fixed interval as a camera captures one image for each intensity variation. The image that best satisfies the criteria for feature extraction is tested on a neural-network based recognition system. The network considers information related to both 2D (contour) and 2.5D shape (local surface curvature) of different objects. Experimental tests performed during different times of the day confirm that the proposed adjustment delivers improved feature extraction, extending the recognition capabilities of the system and adding robustness against changes in ambient light.