Approaches to large-scale structures

Three-dimensional (3D) mapping deals with the problem of building a map of the unknown environments explored by a mobile robot. In contrast to 2D maps, 3D maps contain richer information of the visited places. Besides enabling robot navigation in 3D, a 3D map of the robot surroundings could be of great importance for higher-level robotic tasks, like scene interpretation and object interaction or manipulation, as well as for visualization purposes in general, which are required in surveillance, urban search and rescue, surveying, and others. Hence, the goal of this thesis is to develop a system which is capable of reconstructing the surrounding environment of a mobile robot as a three-dimensional map. Microsoft Kinect camera is a novel sensing sensor that captures dense depth images along with RGB images at high frame rate. Recently, it has dominated the stage of 3D robotic sensing, as it is low-cost, low-power. For this work, it is used as the exteroceptive sensor and obtains 3D point clouds of the surrounding environment. Meanwhile, the wheel odometry of the robot is used to initialize the search for correspondences between different observations. As a single 3D point cloud generated by the Microsoft Kinect sensor is composed of many tens of thousands of data points, it is necessary to compress the raw data to process them efficiently. The method chosen in this work is to use a feature-based representation which simplifies the 3D mapping procedure. The chosen features are planar surfaces and orthogonal corners, which is based on the fact that indoor environments are designed such that walls, ground floors, pillars, and other major parts of the building structures can be modeled as planar surface patches, which are parallel or perpendicular to each other. While orthogonal corners are presented as higher features which are more distinguishable in indoor environment. In this thesis, the main idea is to obtain spatial constraints between pairwise frames by building correspondences between the extracted vertical plane features and corner features. A plane matching algorithm is presented that maximizes the similarity metric between a pair of planes within a search space to determine correspondences between planes. The corner matching result is based on the plane matching results. The estimated spatial constraints form the edges of a pose graph, referred to as graph-based SLAM front-end. In order to build a map, however, a robot must be able to recognize places that it has previously visited. Limitations in sensor processing problem, coupled with environmental ambiguity, make this difficult. In this thesis, we describe a loop closure detection algorithm by compressing point clouds into viewpoint feature histograms, inspired by their strong recognition ability. The estimated roto-translation between detected loop frames is added to the graph representing this newly discovered constraint. Due to the estimation errors, the estimated edges form a non-globally consistent trajectory. With the aid of a linear pose graph optimizing algorithm, the most likely configuration of the robot poses can be estimated given the edges of the graph, referred to as SLAM back-end. Finally, the 3D map is retrieved by attaching each acquired point cloud to the corresponding pose estimate. The approach is validated through different experiments with a mobile robot in an indoor environment

We investigate the deployment of a submersible platform with stereovision imaging capability for three-dimensional (3D) mapping of benthic habitats and other sea-floor structures over local areas. A complete framework is studied, comprising: 1) suitable trajectories to be executed for data collection; 2) data processing for positioning and trajectory followed by online frame-to-frame and frame-to-mosaic registration of images, as well as recursive global realignment of positions along the path; and 3) 3D mapping by the fusion of various visual cues, including motion and stereo within a Kalman filter. The computational requirements of the system are evaluated, formalizing how processing may be achieved in real time. The proposed scenario is simulated for testing with known ground truth to assess the system performance, to quantify various errors, and to identify how performance may be improved. Experiments with underwater images are also presented to verify the performance of various components and the overall scheme.

Surface rendering of three-dimensional myocardial SPECT: Clinical usefulness compared with bull's-eye and conventional tomograms

Dose tolerance at helium and nitrogen temperatures for whole cell electron tomography

Electron Tomography of Ice-Embedded Prokaryotic Cells

Size distributions of cellulose nanocrystals (CNCs), extracted from softwood pulp via strong sulfuric acid hydrolysis, exhibit large variability when analyzed from transmission electron microscopy (TEM) images. In this article, the causes of this variability are studied and discussed. In order to obtain results comparable with those reported, a reference material of CNCs (CNCD-1) was used to evaluate size distribution. CNC TEM specimens were prepared as-stained and dried with a rapid-flushing staining method or hydrated and embedded in vitreous ice with the plunge-freezing method. Several sets of bright-field TEM (BF-TEM), annular dark-field scanning TEM (ADF-STEM) and cryogenic-TEM (cryo-TEM) images were acquired for size distribution analysis to study the contributing factors. The rapid-flushing staining method was found to be the most effective for contrast enhancement of CNCs, not only revealing the helical structure of single CNCs but also resolving the laterally jointed CNCs. During TEM specimen preparation, CNCs were fractionated on TEM grids driven by the coffee-ring effect, as observed from contrast variation of CNCs with a stain-depth gradient. From the edge to the center of the TEM grids, the width of CNCs increases, while the aspect ratio (length to width) decreases. This fractionated dispersion of CNCs suggests that images taken near the center of a droplet would give a larger mean width. In addition to particle fractionation driven by the coffee-ring effect, the arrangement and orientation of CNC particles on the substrate significantly affect the size measurement when CNC aggregation cannot be resolved in images. The coexistence of asymmetric cross-section CNC particles introduces a large variation in size measurement, as TEM images of CNCs are mixed projections of the width and height of particles. As a demonstration of how this contributes to inflated size measurement, twisted CNC particles, rectangular cross-section particles and end-to-end jointed CNCs were revealed in reconstructed three-dimensional (3D) micrographs by electron tomography (ET).

Cryo-electron microscopy is becoming an increasingly powerful tool for solving the structure of protein complexes and has the potential to address problems that cannot be solved using other methods. The field however suffers from several major disadvantages related to the time required to acquire, process and analyze the data and the tedium of using the current prevailing methods. We have for some time been working towards the goal of developing a system that will result in a 3D map of a macromolecular structure automatically and within hours of inserting a specimen into a transmission electron microscope. We propose that these automated methods for data collection and analysis will have a significant impact in transferring the cryo-electron microscopy technology to the general biological community as well as in increasing the volume of data that can be collected during a single session at the microscope. The Leginon system that we have developed is designed to emulate all of the decisions and actions of a highly trained microscopist in collecting data from a vitreous ice specimen. These include identifying suitable areas of vitreous ice at low magnification, determining the presence and location of specimen on the grid, automatically adjusting imaging parameters (focus, astigmatism) under low dose conditions and acquiring images at high magnification to either film or a digital camera.

This study aimed to explore how a three-dimensional (3D) map affects the acquisition of survey knowledge in complex virtual environments (VEs). Subject navigated in virtual and real subway stations, where the virtual station was augmented with either a 3D floor map, a 3D building map, or neither. After subject's navigation in the station, they made the judgment of spatial relative directions between objects. The analyses of response time showed that participants responded faster when they were assisted by the 3D floor or building map than without a map aid in virtual stations. There was no significant difference between survey knowledge acquired in the virtual station augmented with the 3D floor or building map and in the real station. Participants with the 3D floor map performed best in spatial judgments. Implications of these findings to a 3D map design were discussed.

IoT-based low-cost 3D mapping using 2D Lidar for different materials

Electron tomography is the most widely applicable method for obtaining three-dimensional information by electron microscopy. It is, in fact, the only method suitable for investigating pleomorphic structures, such as many supramolecular assemblies, organelles and cells. With the recent development of automated low-dose data-acquisition schemes, it is now possible to study molecules and cells embedded in vitreous ice. This opens up new horizons for investigating the functional organization of cellular components with minimal perturbation of the cellular context. In the present study we used automated electron tomography in conjunction with cryomicroscopy to reveal the internal organization and ultrastructure of mitochondria. The whole isolated cell organelles from Neurospora crassa were quick-frozen and examined in vitrified ice. Single-axis tilt series were recorded with a Philips CM 120 Biotwin under low-dose conditions; the estimated total exposure was 10,000 e−nm−2. The tilt increment was 1° and the tilt series ranged from −65° to +65°. In order to enhance the contrast of the rather thick samples zero-loss energy filtering was employed. The images were recorded with a Ik × Ik CCD camera. The experimental setup has been described by Koster et al2 Three-dimensional reconstructions were performed by weighted backprojection. Prior to the three-dimensional visualization of 3-D data sets a denoising technique based on nonlinear anisotropic diffusion was applied (Figs. 2, 3).

Cryogenic single-particle photoluminescence (PL) spectroscopy has been used with great success to directly observe the heterogeneous photophysical states present in a population of luminescent particles. Cryogenic electron tomography provides complementary nanometer scale structural information to PL spectroscopy, but the two techniques have not been correlated due to technical challenges. Here, we present a method for correlating single-particle information from these two powerful microscopy modalities. We simultaneously observe PL brightness, emission spectrum, and in-plane excitation dipole orientation of CdSSe/ZnS quantum dots suspended in vitreous ice. Stable and fluctuating emitters were observed, as well as a surprising splitting of the PL spectrum into two bands with an average energy separation of 80 meV. In some cases, the onset of the splitting corresponded to changes in the in-plane excitation dipole orientation. These dynamics were assigned to structures of individual quantum dots and the excitation dipoles were visualized in the context of structural features.

Both light and electron microscopy have experienced exciting advances in the past few years, driven by the development of new imaging technologies and the ready availability of powerful computers and associated algorithms for visualization, analysis, and image restoration. The concomitant development of molecular genetic methods for engineering fluorescent molecules into proteins and for targeting indicator dyes to subcellular compartments has led to unprecedented opportunities for the study of dynamic processes and for correlating physiology and structure. Although the effective resolution of light microscopy has been increased through the use of optical sectioning and computational deblussing of images, the resolution of the light microscope is still not sufficient to address Inany issues of cell and tissue structure and protein localization. Electron microscopy has also undergone a quiet revolution, in particular high voltage electron microscopy, where the technique of electron tomography allows 3D reconstruction of relatively large structures such as neuronal spiny dendrites without the need for serial sectioning. In combination with light microscopy, high voltage EM and electron tomography provide a relatively straightforward method to bridge the resolution gap between light and electron microscopy. In this review, we discuss correlated 3D light and electron microscopy imaging of cells and tissues, including the adaptation of fluorescent dyes for electron microscopy imaging. (The J Histotechnol 23:261, 2000)

To introduce and investigate a method for free-breathing three-dimensional (3D) mapping of the human body at ultrahigh field (UHF), which can be used to generate homogenous flip angle (FA) distributions in the human body at UHF. A 3D relative mapping sequence with a radial phase-encoding (RPE) k-space trajectory was developed and applied in 11 healthy subjects at 7T. An RPE-based actual flip angle mapping method was applied with a dedicated shim setting to calibrate the relative maps yielding absolute maps of the individual transmit channels. The method was evaluated in a motion phantom and by multidimensional in vivo measurements. Additionally, 3D gradient echo scans with and without static phase-only shims were used to qualitatively validate shim predictions. The phantom validation revealed good agreement for maps between dynamic measurement and static reference acquisition. The proposed 3D method was successfully validated in vivo by comparing magnitude and phase distributions with a 2D Cartesian reference. 3D maps free from visible motion artifacts were successfully acquired for 11 subjects with body mass indexes ranging from 19 kg/m2 to 34 kg/m2 . 3D respiration-resolved absolute maps indicated FA differences between inhalation and exhalation up to 15% for one channel and up to 24% for combined channels for shallow breathing. The proposed method provides respiration-resolved absolute 3D maps of the human body at UHF, which enables the investigation and development of 3D shimming and parallel transmission methods to further enhance body imaging at UHF.

We investigate structural order in glassy water by performing classical molecular dynamics simulations using the extended simple point charge (SPC/E) model of water. We perform isochoric cooling simulations across the glass transition temperature at different cooling rates and densities. We quantify structural order by orientational and translational order metrics. Upon cooling the liquid into the glassy state, both the orientational order parameter Q and translational order parameter tau increase. At T=0 K, the glasses fall on a line in the Q-tau plane or order map. The position of this line depends only on density and coincides with the location in the order map of the inherent structures (IS) sampled upon cooling. We evaluate the energy of the IS, eIS(T), and find that both order parameters for the IS are proportional to eIS. We also study the structural order during the transformation of low-density amorphous ice (LDA) to high-density amorphous ice (HDA) upon isothermal compression and are able to identify distinct regions in the order map corresponding to these glasses. Comparison of the order parameters for LDA and HDA with those obtained upon isochoric cooling indicates major structural differences between glasses obtained by cooling and glasses obtained by compression. These structural differences are only weakly reflected in the pair correlation function. We also characterize the evolution of structural order upon isobaric annealing, leading at high pressure to very-high density amorphous ice (VHDA).

Knowledge of three-dimensional (3D) structures of each individual particles of asymmetric and flexible proteins is essential in understanding those proteins’ functions; but their structures are difficult to determine. Electron tomography (ET) provides a tool for imaging a single and unique biological object from a series of tilted angles, but it is challenging to image a single protein for three-dimensional (3D) reconstruction due to the imperfect mechanical control capability of the specimen goniometer under both a medium to high magnification (approximately 50,000–160,000×) and an optimized beam coherence condition. Here, we report a fully mechanical control method for automating ET data acquisition without using beam tilt/shift processes. This method could reduce the accumulation of beam tilt/shift that used to compensate the error from the mechanical control, but downgraded the beam coherence. Our method was developed by minimizing the error of the target object center during the tilting process through a closed-loop proportional-integral (PI) control algorithm. The validations by both negative staining (NS) and cryo-electron microscopy (cryo-EM) suggest that this method has a comparable capability to other ET methods in tracking target proteins while maintaining optimized beam coherence conditions for imaging.