Development of a Goniometer to capture 3D models using Pi Noi camera

In Amazon Robotic Challenge, we need to make unknown object's 3d model for recognizing and estimating 6D pose in limited time. The paper presents a unique autonomous system to build object's complete 3d model fast. This method is different with others for it can build object's model contains object's bottom side. We modify ElasticFusion to focus on building object's 3D model from object's up-side with assistant pose information from Aruco Markers. Then instead of using ICP or other merge algorithms, we merge object's upside point cloud and bottom-side point cloud easily by using Aruco Markers on transparent board. At last, we use octree and source point cloud to color poisson reconstruction mesh model to obtain object's final complete colored 3d model. Experiment results show that our approach can generate common object's complete model with acceptable size error in limited time.

Manual data annotation for training custom object detection can be a time-consuming and error-prone process. In this paper, we propose an automatic approach to generating synthetic, annotated images using differentiable neural rendering and 3D object models. We also investigate the possibility of using 3D adversarial object models to improve object detection accuracy. The experimental results show that the object detection models trained using both synthetic examples rendered from 3D object models and real data outperform the baseline model trained on only real data.

We have been studying an in-situ 3D modeling and authoring system, AR Diorama. In the AR Diorama system, a user is able to reconstruct a 3D model of a real object of concern and describe behaviors of the model by stroke input. In this article, we will introduce two ongoing studies on interactive 3D reconstruction techniques. First technique is feature-based. Natural feature points are first extracted and tracked. A convex hull is then obtained from the feature points based on Delaunay tetrahedralisation. The polygon mesh is carved to approximate the target object based on a feature-point visibility test. Second technique is region-based. Foreground and background color distribution models are first estimated to extract an object region. Then a 3D model of the target object is reconstructed by silhouette carving. Experimental results show that the two techniques can reconstruct a better 3D model interactively compared with our previous system.

Robot localization is one of the most common tasks in autonomous systems. Unlike most vision-based methods which perform localization from cameras mounted on the robot, we propose a system which uses an off-board camera to achieve localization. For this purpose, we use a similarity measure between a camera image and a synthetic image generated using a 3D object model. In contrast to other methods, a previous training period is not necessary as we use a model of shading appearance based on the surface curvature of the 3D model. Assuming a reasonably planar area which is observed by an off-board camera, an initial position estimation in the camera image (based on 2D blob tracking) makes it possible for the 3D model to be rendered near the object in the image. From this initial guess, the model is then compared against the real image, such that it can converge to the true vehicle pose. Experiments are performed with a forklift-like robot in an outdoor industrial environment, considering different illumination conditions. The results are compared against a laser-based ground-truth, and illustrate the applicability of the method with a low average error.

The authors present the results of theoretical and experimental studies on the use of laser scanning technology for the 3D-modeling of real estate objects in the cadastre. The result of the theoretical part of the research was a set of parameters for 3D modeling of real estate objects influencing the choice of the scanning method (ground, air or mobile), which showed that when creating separate 3D models of real estate objects (buildings, structures, premises and objects of unfinished construction) the method of ground-based laser scanning should be used. Based on the theoretical part, an experiment on 3D-modeling of a typical real estate object (building) was performed on the basis of data obtained with the method of ground- based laser scanning. The model of the real estate object was built with various degrees of detail to demonstrate its accuracy characteristics, depending on the distance from the scanner stations, the use of photographic materials, etc. The results of the experimental part of the studies show that the model obtained is in full conformity with the current requirements of the legislation in the field of cadastre. The proposed solution can be used as a methodological basis for constructing three-dimensional models in the transition of the Russian Federation to a 3D-cadastre system.

Computer vision systems can be used to determine the shapes of real three-dimensional objects for purposes of object recognition and pose estimation or for CAD applications. One method that has been developed is photometric stereo. This method uses several images taken from the same viewpoint, but with different lightings, to determine the three-dimensional shape of an object. Most previous work in photometric stereo has been with gray-tone images; color images have only been used for dielectric materials. In this paper we describe a procedure for color photometric stereo, which recovers the shape of a colored object from two or more color images of the object under white illumination. This method can handle different types of materials, such as composites and metals, and can employ various reflection models such as the Lambertian, dichromatic, and Torrance-Sparrow models. For composite materials, colored metals, and dielectrics, there are two advantages of utilizing color information: at each pixel, there are more constraints on the orientation, and the result is less sensitive to noise. Consequently, the shape can be found more accurately. The method has been tested on both artificial and real images of objects of various materials, and on real images of a multi-colored object.

The essential techniques used in 3D model reconstruction are LIDAR scanning, close-range photogrammetry, and aerial photogrammetry. The aim of this study is to investigate 3D building model results using a UAV platform using different altitudes and ranges. The methodology was divided into four phases. Phase one was a preliminary study and phase two was the data acquisition phase which included flight planning, camera calibration, and image acquisition at three different altitudes; 150m, 170m, and 190m and at different ranges; 180m, 190m, and 200m, respectively. Phase three was data processing whereby the resulting 3D models were generated using five steps which are image masking, image aligning, image dense clouding, image meshing, and image texturing. Phase four contained analyses on the 3D model such as accuracy assessment of the 3D model and analysis on the different altitudes and ranges employed. This study stated that the maximum accuracy was 0.195m without a control point, while the minimum accuracy was 0.123m with a control point. It shows that the 3D model with a control point produced measurements that were 37% more accurate than the 3D model without a control point. In the final analysis, the results showed that the built 3D model resulting from the UAV platform can provide an accuracy of less than 0.200m with or without control points. These findings can be applied to many fields, such as heritage and conservation in the efforts to preserve the 3D model of heritage objects.

An approach to automatically extract three dimensional (3D) models (that is, geometries and topologies) of physical objects in a facility is described. The rationale for this work is its repeated use in efficiently developing databases of 3D objects for applying virtual reality (VR) tools in detailed layout decision support. Obtaining 3D object models can be a challenging task. Sometimes they are available, for example, in a Computer-Aided Design (CAD) database and these can be readily imported into a VR database. But on many occasions one is not so fortunate and these object models have to be created in correlation to an existing or proposed facility, which can be an extremely tedious and time consuming task. A time efficient and economical alternative is to use video camera images, but quickly and accurately capturing the depth information from 2D camera images has so far remained elusive because the existing methodologies are too general purpose and operate at a lower level of abstraction, namely digitized images. We have developed a method for directly inputting 3D objects into VR-aided facility layout models, by integrating the strengths of previously tried and tested technological components: (i) camera calibration; (ii) image processing; (iii) stereo vision; and (iv) Delaunay triangulation. The techniques described here are embedded in a prototype architecture and toolkit called MIRRORS (Methodology for Inputting Raw Recordings into 3D Object Renderings for Stereo). The primary contribution of this paper is that it has been able to design an integrated system to build 3D object models from 2D images. The MIRRORS system has been primarily designed for objects without free-form surfaces and whose shape can be recovered from a relatively nondense set of points.

Previous studies have found that preschoolers are confused about the relationship between two‐dimensional (2D) symbols and their referents. Preschoolers report that 2D images (e.g. televised images and photographs) share some of the characteristics of the objects they are representing. A novel Comparison Task was created to test what might account for these misattributions of the properties of 2D images. Three‐and‐a‐half‐ and four‐and‐a‐half‐year olds made comparisons between items presented in two of three formats simultaneously, i.e. as a real object and as an object appearing on television (TV), as a real object and as a photo of that object, or as an object appearing on TV and in a photo. Presenting the televised object or the photo of the object along with the real object aided performance; children indicated that televised objects and photos of objects no longer shared the properties of the real objects. Children also performed better when presented with two 2D images of the same object (the photo of the object and the televised image of the object). Although dual‐representation might account for these misattributions of the properties of 2D images, because children also performed better when two 2D images of the same object were present, other possibilities such as children using pictures as communicative devices and disregarding the pragmatics of the task should be explored. Copyright © 2010 John Wiley & Sons, Ltd.

An algorithm for the robust and fast automatic construction of a 3D model of any real object using images from multiple views is presented. The images are taken from a real object rotating in front of a stationary calibrated CCD TV camera. The object silhouettes extracted from the input images, the related turntable positions, and camera orientation are used to construct the volume model of the real object by applying the method of occluding contours. A keypoint in performing this method is a proper volume representation, characterized by low complexity and suitability for a fast computation of volume models. In the presented approach, each volume model is described by pillar-like volume elements (pillars) ensuring a computational complexity proportional to the size of the real object surface and enabling a fast and simple construction of the volume model. The fast performance is due to the simple projection feasibility of those pillars and the easy-to-perform intersection test for the object silhouette with the projected pillars. Results with real image sequences have confirmed the robustness of the developed algorithm even for the modelling of real objects with highly detailed and complex surfaces and the use of imperfect object silhouettes.

Both 3D models and 2D images contain a wealth of information about everyday objects in our environment. However, it is difficult to semantically link together these two media forms, even when they feature identical or very similar objects. We propose a joint embedding space populated by both 3D shapes and 2D images of objects, where the distances between embedded entities reflect similarity between the underlying objects. This joint embedding space facilitates comparison between entities of either form, and allows for cross-modality retrieval. We construct the embedding space using 3D shape similarity measure, as 3D shapes are more pure and complete than their appearance in images, leading to more robust distance metrics. We then employ a Convolutional Neural Network (CNN) to "purify" images by muting distracting factors. The CNN is trained to map an image to a point in the embedding space, so that it is close to a point attributed to a 3D model of a similar object to the one depicted in the image. This purifying capability of the CNN is accomplished with the help of a large amount of training data consisting of images synthesized from 3D shapes. Our joint embedding allows cross-view image retrieval, image-based shape retrieval, as well as shape-based image retrieval. We evaluate our method on these retrieval tasks and show that it consistently out-performs state-of-the-art methods, and demonstrate the usability of a joint embedding in a number of additional applications.

We propose a new approach to building appearance models of 3D objects which is based on Canonical Correlation Analysis (CCA). In appearance based modeling, instead of building an explicit object model (e.g., 3D geometrical object model), a low dimensional object representation is obtained from a set of images. In standard appearance models typically Principal Component Analysis (PCA) is used for feature extraction. In our experiments we compare the performance of standard appearance models based on PCA and models based on CCA for 3D pose estimation. Results indicate that, while getting by with a smaller number linear features, CCA-based models perform consistently better.

It is important to enable a robot to manipulate a target object that has no 3-D model information and is situated in an environment with other unknown objects nearby. This poses an open problem of how to combine perception and manipulation to enable the robot to build an appearance-based model of the target object on the spot to facilitate further manipulation of the object while avoiding the other unknown obstacles in the way. In this paper, we introduce an approach to enable a continuum manipulator, which is apt to maneuver through a crowded environment, to gradually build a 3-D surface model of the target object by moving an RGB-D sensor around the object while also detecting and avoiding surrounding unknown obstacles. Our approach interleaves perception and manipulation such that perception guides the manipulator movement, which in turn allows more perception of the target object for object model building and further manipulator motion. Our approach is characterized by a progressive strategy to register RGB-D images of the target object to build and extend a partial model of the object and the corresponding motion planning strategy for the continuum robot to carry out model building and avoid obstacles at the same time. To demonstrate the effectiveness of our approach, experiments on progressive model building of real objects from real RGB-D images are conducted, where a simulated continuum robot plans and executes its motion to carry the RGB-D camera around a target object for taking those images in an augmented reality setting.

The purpose of the present study is to combine the two theories of Shabah al-Mohaki (the representative image), and Wojud al-Zehni (the mental existence). In the context of Islamic philosophy, the relationship between mental and external realities has been a controversial issue among philosophers. It is well known that proponents of Shabah al-Mohaki believe that there is a dissimilarity between mental and external realities, and their rivals, the proponents of mental existence, believe that there is an essential identity between mental and external realities. This group claims that objects have two types of existence; "objective" which is the same reality in the external world, and “subjective” which is called "mental existence". It seems that in their view, subjective and objective realities are numerically identical. The representative theorists on the other side reject this claim. By investigating and analyzing the texts of the two theories, the author found that the two speak of identity but qualitative and not numerical identity. As far as the mental existence theory is concerned, when we analyze the claim of mental existence theorists and numerical identity, it turns out that they do not say anything better and more than what representative theorists claim. This analysis is done through the two principles of indiscernibility of identicals and identity of indiscernibles. According to the first, if two things are identical then they must be not be different from one another with respect to all of their properties. The principle speaks of numerical identity. The second principle (although it is controversial) states that if two things have the same properties in every respect then they are identical. This principle does not have the logical truth and speaks of qualitative identity. If one considers the case of mental existence, the claim that mental existence and objective reality are numerically identical, then he identifies that these two are not only identical but they have different properties. For instance, the subjective reality such as the image of a book or fire has the property of being mental and the objective reality of that image, the book or fire in the external world, has the property of being external. This shows that they are two different things because they have different properties. Therefore, the only way to say that these two are the same (identical), is that they are qualitatively but not numerically, identical. On the other side, the identity claim (qualitative identity) between subjective and objective realities is found in the texts of proponents of representative theorists. For instance, Shamsuddin Isfahani argues that what is ascribed to mental existence is an image of objective reality and the image of that particular object is contrary to the objective reality itself, although in some respects it is identical to that external reality in such a way that what is obtained from the object in the mind is the same as the external reality. This shows that representative theorists also accept the claim of identity in some respect but qualitative and not numerical identity between subjective and objective reality.

The software platform for mathematical modeling support Temetos is decribed. The Temetos platform is the integrated graphical tool environment for the study of mathematical models of physical processes and technical systems. Temetos platform provides opportunities for preparation of geometric and physical models of the object; external or built-in problem-oriented modules configuration, including numerical method parameters configuration; computational module launch and executing control; calculations results visualization and analysis. The platform consists of three subsystems: the mathematical subsystem, including command-line utilities for the computational domain discretization and triangulation and software modules for the numerical solution of the mathematical model differntial equation systems; instrumental graphics subsystem, including tools for the 3D geometric design of the investigated object model; subject libraries subsystem containing structured knowledgebase for the simulated structures, constructions and physical processes (elements of the libraries are: 3D object models, meshes, material physical models and others.). The subject libraries subsystem is the base of platform information structure. It contains of free format XML files decribing knowledgebase elements. Hierarchical library architecture and universal data interfaces leads the platform to grow and to implement the new subject areas and mathematical models in easy way. As examples of platform implementation the simulation of plasma flow instability in astrophysical conditions and model of brittle material failure under heat load are considered.