Three-dimensional Face Reconstruction using 3D Morphable Model Fitting Method

Photogrammetric surveying using unmanned aerial systems (UAS) and structure from motion (SfM) is a rapidly emerging technique that has been widely used as a valuable research tool in geodesy and related disciplines. We studied surveying and three-dimensional (3D) modeling based on UAS and SfM. In particular, we focus on the accuracy and quality of UAS-SfM in 3D modeling and volumetric calculation of a building with an overhanging feature. For this purpose, two aerial surveys were performed using a fixed-wing and a multirotor aircraft with nadir and oblique camera axes. For validating the results, field surveys were carried out using terrestrial laser scanner (TLS), differential global navigation satellite system, and total station. To study the performance of SfM software on the accuracy and quality of point clouds, UAS images were processed using three software packages: Agisoft PhotoScan, Pix4Dmapper, and 3Dsurvey. As a result of photogrammetric image processing, 3D dense point clouds, mesh models, and digital surface models were generated. For validating the UAS-SfM results, the models were compared with the TLS model and field measurements; absolute accuracy assessments and model versus model comparisons were performed. The most accurate and high-quality 3D models comparable to TLS models were obtained from oblique images. PhotoScan generated relatively more accurate 3D models with the horizontal and vertical accuracy of root mean square error 0.025 and 0.02 m, respectively. The maximum discrepancies in dimensions and volume measurements from the UAS-SfM-derived point cloud were at the level of 0.01 m and 0.66 m3 (0.03%). The results highlighted that using a low-cost UAS and SfM photogrammetry, high-accuracy and high-quality 3D models can be generated. The results confirm that with respect to its rapid data acquisition and centimeter-level accuracy, UAS-SfM can be considered as a suitable alternative approach to conventional measurement techniques, especially in engineering and emergency response.

Nowadays, 3D modelling has become very important because real information can be extracted from the model. UAV is system that was used for developing the 3D model and has proven capable to produce good results. However, the exploration of the integration of UAV with other technology like LiDAR and MLS to develop a more accurate and detailed model is still lacking. This study reviews the aptitudes of the UAV integrated with LiDAR and MLS for developing the 3D building model in Malaysia. Several issues, like type of platform, quality of 3D model and others to review the capabilities of the UAV used for producing very accurate 3D model were critically analysed. Previous research suggests that integration of Lidar with multi-rotor platform fusing with MLS could be used to construct an accurate 3D model. A dense point cloud of the whole building can be obtained from the fusion between Lidar and MLS. Moreover, an accurate 3D model can be generated from this process as the point-cloud from LiDAR and MLS has positional accuracy in centimetre level compared to a non-metric camera integrated with a UAV. This study finds the UAV has the potential to use to produce a quality 3D model.

Fast Multi-View 3D reconstruction of seedlings based on automatic viewpoint planning

Proceedings Volume 9535, Third International Conference on Remote Sensing and Geoinformation of the Environment (RSCy2015)

The three-dimensional (3D) morphological structure of wheat directly reflects the interrelationship among genetics, environments, and cropping systems. However, the morphological complexity of wheat limits its rapid and accurate 3D modelling. We have developed a 3D wheat modelling method that is based on the progression from skeletons to mesh models. Firstly, we identified five morphological parameters that describe the 3D leaf features of wheat from amounts of 3D leaf digitizing data at the grain filling stage. The template samples were selected based on the similarity between the input leaf skeleton and leaf templates in the constructed wheat leaf database. The leaf modelling was then performed using the as-rigid-as-possible (ARAP) mesh deformation method. We found that 3D wheat modelling at the individual leaf level, leaf group, and individual plant scales can be achieved. Compared with directly acquiring 3D digitizing data for 3D modelling, it saves 79.9% of the time. The minimum correlation R2 of the extracted morphological leaf parameters between using the measured data and 3D model by this method was 0.91 and the maximum RMSE was 0.03, implying that this method preserves the morphological leaf features. The proposed method provides a strong foundation for further morphological phenotype extraction, functional–structural analysis, and virtual reality applications in wheat plants. Overall, we provide a new 3D modelling method for complex plants.

A novel method for simultaneously acquiring and registering range data of a real object from different viewpoints is presented. Currently, most 3D model reconstruction techniques do not cooperate with the existing range data for shape recovery of future viewpoints. In this work, a stereo vision system is developed for 3D model acquisition. To reduce the computation and increase the accuracy of stereo matching algorithms, the recovered range data from previous viewpoints are registered and then used to provide additional constraints for 3D acquisition of the next viewpoint. Experiments have shown that our approach gives better performance on both execution time and stereo matching results.

Recovering a complete, detailed, accurate and realistic 3D model from images is still a difficult task. This paper proposes an improved 3D modeling using multistage feature extraction and matching. A 3D modeling technique is applied based on the features extracted from images of the same scene taken from different directions. Matching of the images are used to construct the 3D model. Using multistage features extraction and matching algorithms, multiple 3D models can be obtained. This paper will focus on modeling from reality rather than computer graphics creation of artificial world models. Experimental results are illustrated to explore the effectiveness of the proposed method.

The utilization of customized three-dimentional (3D) models based on patient's computed tomography (CT) scan data and by assistance of additive manufacturing/rapid prototyping (AM/RP) techniques for 3D reconstruction is one of the applicable trends for reducing the errors and time saving during surgeries. In the present study, the methodology of the fabrication of a custom-made 3D model based on converting CT scan data to standard triangle language (STL) format for a 33-years old male patient who was suffering from an accident trauma was described. The 3D model of the patient’s skull was fabricated and applied in preoperative planning. It was used for designing a comprehensive plan for rehabilitation of the damaged orbit to restore the appearance and bone reconstruction of the patient. Before fabricating the model, the accuracy of protocols used in converting CT scan data into STL file was evaluated. Then, the model was fabricated by a fused deposition modeling (FDM) machine. Using this procedure led to a maximum of 1.4% difference between the virtual model in the software and the fabricated 3D model in the fracture site. The present technique reduced operation time significantly. In addition, following eight months from the operation, the treatment approach ensured the patient's fractures healing process.

The objective of this study was to create a real model of a face using the well preserved "Bochdalek's skull" (from an eighteenth Century female aged 18 years) kept in the museum of anatomy (Institute of Anatomy, 1st Medical Faculty, Charles University in Prague). The skull had previously been appraised as a deformed skull with an adhesion present on both sides of the jaw, most likely of post-traumatic origin (bilateral syngnathia). In an attempt to find the best description for it, and to identify the spatial relationships between the surface of the facial bones which had changed in shape, as well as the formation of soft tissue on the face, we decided to perform a 3D reconstruction of the face. Due to the necessity of preserving the unique original undamaged skull, we created an exact digital "casting" of the facial bone structure on a computer first, which we then converted into a three-dimensional model using a 3D RepRap printer. We needed to take into consideration the fact that we had no portrait of the girl, just the skull. For this reason, we opted for a selected combination of anthropologic steps (the modified Manchester technique), which in our view, allows for optimum creation of the topography of the face in keeping with the deformed skull. The resulting reconstructed face was old in appearance with an overhanging lower lip and flattened surfaces in the areas of the temporalis and masseter muscles.

Many archeological sites have been destroyed totally or partially around the globe due to earth quakes, floods and/or fire. It is the need of time to digitally preserve the historical sites before they get destroyed. It is difficult to obtain data using conventional methods due to difficulty in accessing the historical sites because of their location and neighborhood. Recent innovations in Unmanned Aerial Systems (UASs) have enabled to remote sense and acquire data using aerial imagery. In this research photogrammetry is used to develop highly accurate photo realistic 3D models. This method contributes in terms of quality, time and investment to acquire data compared to other methods which demand more time and investment. The automated flight stability and altitude control of the UAS and the fully stabilized and calibrated RGB sensor helps deliver high quality data to provide reliable results. Using aerial data acquired from a low cost and light weight Unmanned Aerial Vehicles (UAV) equipped with high resolution RGB sensor and image processing using photogrammetry tools highly accurate 3D model is formed. The main goals of this study were to conduct survey of a historical site in Pakistan utilizing advanced robotics and photogrammetry stitching techniques to investigate the potential of UAV in producing geo-referenced 3D reconstruction. The study is carried out to achieve an accuracy of within 5cm/pixel of the 3D model. The digitally preserved 3D models will help to maintain and reconstruct and historical site in case of any damage. The 3D model produced can be quickly produced for inventory, architectural and facility management system.

A computer modeling system for facial reconstruction has been developed that employs a touch-based application to create anatomically accurate facial models focusing on skeletal detail. This article discusses the advantages and disadvantages of the system and illustrates its accuracy and reliability with a blind study using computed tomography (CT) data of living individuals. Three-dimensional models of the skulls of two white North American adults (one male, one female) were imported into the computer system. Facial reconstructions were produced by two practitioners following the Manchester method. Two posters were produced, each including a face pool of five surface model images and the facial reconstruction. The face pool related to the sex, age, and ethnic group of the target individual and included the surface model image of the target individual. Fifty-two volunteers were asked to choose the face from the face pool that most resembled each reconstruction. Both reconstructions received majority percentage hit rates that were at least 50% greater than any other face in the pool. The combined percentage hit rate was 50% above chance (70%). A quantitative comparison of the facial morphology between the facial reconstructions and the CT scan models of the subjects was carried out using Rapidform™ 2004 PP2-RF4. The majority of the surfaces of the facial reconstructions showed less than 2.5 mm error and 90% of the male face and 75% of the female face showed less than 5 mm error. Many of the differences between the facial reconstructions and the facial scans were probably the result of positional effects caused during the CT scanning procedure, especially on the female subject who had a fatter face than the male subject. The areas of most facial reconstruction error were at the ears and nasal tip.

Tractography of the facial nerve based on single-shell diffusion MR imaging is thought to be helpful before surgery for resection of vestibular schwannoma. However, this paradigm can be vitiated by the isotropic diffusion of the CSF, the convoluted path of the facial nerve, and its crossing with other bundles. Here we propose a multishell diffusion MR imaging acquisition scheme combined with probabilistic tractography that has the potential to provide a presurgical facial nerve reconstruction uncontaminated by such effects. Five patients scheduled for vestibular schwannoma resection underwent multishell diffusion MR imaging (b-values = 0, 300, 1000, 2000 s/mm2). Facial nerve tractography was performed with a probabilistic algorithm and anatomic seeds located in the brain stem, cerebellopontine cistern, and internal auditory canal. A single-shell diffusion MR imaging (b-value = 0, 1000 s/mm2) subset was extrapolated from the multishell diffusion MR imaging data. The quality of the facial nerve reconstruction based on both multishell diffusion MR imaging and single-shell diffusion MR imaging sequences was assessed against intraoperative videos recorded during the operation. Single-shell diffusion MR imaging-based tractography was characterized by failures in facial nerve tracking (2/5 cases) and inaccurate facial nerve reconstructions displaying false-positives and partial volume effects. In contrast, multishell diffusion MR imaging-based tractography provided accurate facial nerve reconstructions (4/5 cases), even in the presence of ostensibly complex patterns. In comparison with single-shell diffusion MR imaging, the combination of multishell diffusion MR imaging-based tractography and probabilistic algorithms is a more valuable aid for surgeons before vestibular schwannoma resection, providing more accurate facial nerve reconstructions, which may ultimately improve the postsurgical patient's outcome.

Archaeological 3D digitization of skeletal elements is an essential aspect of the discipline. Objectives are various: archiving of data (especially before destructive sampling for biomolecular studies for example), study or for pedagogical purposes to allow their manipulation. As techniques are rapidly evolving, the question that arises is the use of appropriate methods to answer the different questions and guarantee sufficient quality of information. The combined use of different 3D technologies for the study of a single Mesolithic bone fragment from Brittany (France) is here an opportunity to compare different 3D digitization methods. This oldest human bone of Brittany, a clavicle constituted of two pieces, was dug up from the mesolithic shell midden of Beg-er-Vil in Quiberon and dated from ca. 8200 to 8000 years BP. They are bound to post-mortem processing, realized on fresh bone in order to remove the integuments, which it is necessary to better qualify. The clavicle was studied through a process that combines advanced 3D image acquisition, 3D processing, and 3D printing with the goal to provide relevant support for the experts involved in the work. The bones were first studied with a metallographic microscopy, scanned with a CT scan, and digitized with photogrammetry in order to get a high quality textured model. The CT scan appeared to be insufficient for a detailed analysis; the study was thus completed with a µ-CT providing a very accurate 3D model of the bone. Several 3D-printed copies of the collarbone were produced in order to support knowledge sharing between the experts involved in the study. The 3D models generated from µCT and photogrammetry were combined to provide an accurate and detailed 3D model. This model was used to study desquamation and the different cut marks, including their angle of attack. These cut marks were also studied with traditional binoculars and digital microscopy. This last technique allowed characterizing their type, revealing a probable meat cutting process with a flint tool. This work of crossed analyses allows us to document a fundamental patrimonial piece, and to ensure its preservation. Copies are also available for the regional museums.

India is a country rich in regionally diversified cultural heritage monuments and ranked as sixth in number according to UNESCO cultural heritage sites. Heritage monument site is a location where there is importance in terms of social history that is preserved based upon its rich cultural values. Warangal is one of the Indian cities rich in heritage monuments. It is also included in the country’s prestigious project HCDAY (Heritage City Development and Augmentation Yojana) monuments for restoration and preservation. Kota Gullu comprises about 20 temples, and all of them are different in design and size that portray architectural values that take back us to the twelfth century. UAV is used in acquiring aerial photography and collection of monuments imaging data. With the advancements of UAV, high-resolution images are acquired in stipulated time. The sophisticated techniques in photogrammetry also play a crucial role in utilizing photographs for generating relatively accurate 3D models. Kota Gullu structure is chosen in the preservation of the 3D model. Pix4Dcapture is a mobile-based application used for acquiring the images of the temple ruins in the study area. The processing of the images to 3D models is done by using Pix4Dmapper Desktop software. This case study develops a 3D model which can be used for preservation of the heritage monument.

A 3D model of a particle can be obtained using multiple-view silhouettes. Using the 3D model we can estimate the volume of the particle together with other shape features such as the elongation, compactness and flatness. However, there may be a large volume estimation error due to an insufficient distribution of particle views. The error can be reduced by merging multiple captures of the particle with the particle in different orientations, yielding a more accurate 3D model. This is a time-consuming process. Therefore, we investigate the use of shape features to predict a volume scaling factor which when applied result in an improved mean volume estimation. Shape features extracted from the 3D models are clustered into shape classes using k-means clustering. Then, for each shape class we estimate a volume scaling factor that zeroes the per-class mean volume error. For our test data, this method reduces the single capture mean volume estimation error from 20% to 6.4%. Furthermore, we show that by applying this shape-dependent scaling factor, the number of particle views required to obtain a 2% error is fewer than if we applied an average scaling factor that does not account for the particle shape. Future work will be aimed at exploring additional ways to decrease the volume estimation error.