High-Resolution Optical Microscopy for Bio-Imaging

Far-field optical microscopy using focused light is an important tool in a number of scientific disciplines including chemical, (bio)physical and biomedical research, particularly with respect to the study of living cells and organisms. Unfortunately, the applicability of the optical microscope is limited, since the diffraction of light imposes limitations on the spatial resolution of the image. Consequently the details of, for example, cellular protein distributions, can be visualized only to a certain extent. Fortunately, recent years have witnessed the development of ‘super-resolution’ far-field optical microscopy (nanoscopy) techniques such as stimulated emission depletion (STED), ground state depletion (GSD), reversible saturated optical (fluorescence) transitions (RESOLFT), photoactivation localization microscopy (PALM), stochastic optical reconstruction microscopy (STORM), structured illumination microscopy (SIM) or saturated structured illumination microscopy (SSIM), all in one way or another addressing the problem of the limited spatial resolution of far-field optical microscopy. While SIM achieves a two-fold improvement in spatial resolution compared to conventional optical microscopy, STED, RESOLFT, PALM/STORM, or SSIM have all gone beyond, pushing the limits of optical image resolution to the nanometer scale. Consequently, all super-resolution techniques open new avenues of biomedical research. Because the field is so young, the potential capabilities of different super-resolution microscopy approaches have yet to be fully explored, and uncertainties remain when considering the best choice of methodology. Thus, even for experts, the road to the future is sometimes shrouded in mist. The super-resolution optical microscopy roadmap of Journal of Physics D: Applied Physics addresses this need for clarity. It provides guidance to the outstanding questions through a collection of short review articles from experts in the field, giving a thorough discussion on the concepts underlying super-resolution optical microscopy, the potential of different approaches, the importance of label optimization (such as reversible photoswitchable proteins) and applications in which these methods will have a significant impact.Mark Bates, Christian Eggeling

Widely used MRI methods show brain morphology both in vivo and ex vivo at very high resolution. Many of these methods (e.g., T2*-weighted imaging, phase-sensitive imaging, or susceptibility-weighted imaging) are sensitive to local magnetic susceptibility gradients produced by subtle variations in tissue composition. However, the spectral resolution of commonly used methods is limited to maintain reasonable run-time combined with very high spatial resolution. Here, the authors report on data acquisition at increased spectral resolution, with 3-dimensional high spectral and spatial resolution MRI, in order to analyze subtle variations in water proton resonance frequency and lineshape that reflect local anatomy. The resulting information compliments previous studies based on T2* and resonance frequency. The proton free induction decay was sampled at high resolution and Fourier transformed to produce a high-resolution water spectrum for each image voxel in a 3D volume. Data were acquired using a multigradient echo pulse sequence (i.e., echo-planar spectroscopic imaging) with a spatial resolution of 50 × 50 × 70 μm(3) and spectral resolution of 3.5 Hz. Data were analyzed in the spectral domain, and images were produced from the various Fourier components of the water resonance. This allowed precise measurement of local variations in water resonance frequency and lineshape, at the expense of significantly increased run time (16-24 h). High contrast T2*-weighted images were produced from the peak of the water resonance (peak height image), revealing a high degree of anatomical detail, specifically in the hippocampus and cerebellum. In images produced from Fourier components of the water resonance at -7.0 Hz from the peak, the contrast between deep white matter tracts and the surrounding tissue is the reverse of the contrast in water peak height images. This indicates the presence of a shoulder in the water resonance that is not present at +7.0 Hz and may be specific to white matter anatomy. Moreover, a frequency shift of 6.76 ± 0.55 Hz was measured between the molecular and granular layers of the cerebellum. This shift is demonstrated in corresponding spectra; water peaks from voxels in the molecular and granular layers are consistently 2 bins apart (7.0 Hz, as dictated by the spectral resolution) from one another. High spectral and spatial resolution MR imaging has the potential to accurately measure the changes in the water resonance in small voxels. This information can guide optimization and interpretation of more commonly used, more rapid imaging methods that depend on image contrast produced by local susceptibility gradients. In addition, with improved sampling methods, high spectral and spatial resolution data could be acquired in reasonable run times, and used for in vivo scans to increase sensitivity to variations in local susceptibility.

Hyperspectral images (HSI) have high spectral and low spatial resolutions. However, multispectral images (MSI) usually have low spectral and high spatial resolutions. In various applications HSI with high spectral and spatial resolutions are required. In this paper, a new method for spatial resolution enhancement of HSI using high resolution MSI based on sparse coding and linear spectral unmixing (SCLSU) is introduced. In the proposed method (SCLSU), high spectral resolution features of HSI and high spatial resolution features of MSI are fused. In this case, the sparse representation of some high resolution MSI and linear spectral unmixing (LSU) model of HSI and MSI is simultaneously used in order to construct high resolution HSI (HRHSI). The fusion process of HSI and MSI is formulated as an ill-posed inverse problem. It is solved by the Split Augmented Lagrangian Shrinkage Algorithm (SALSA) and an orthogonal matching pursuit (OMP) algorithm. Finally, the proposed algorithm is applied to the Hyperion and ALI datasets. Compared with the other state-of-the-art algorithms such as Coupled Nonnegative Matrix Factorization (CNMF) and local spectral unmixing, the SCLSU has significantly increased the spatial resolution and in addition the spectral content of HSI is well maintained.

Improved efficiency of high resolution thermal and cold neutron imaging

Most modern satellites can provide two types of images: 1) panchromatic (PAN) image and 2) multispectral (MS) image. The former has high spatial resolution and low spectral resolution, while the latter has high spectral resolution and low spatial resolution. To obtain images with both high spectral and spatial resolution, pansharpening has emerged to fuse the spatial information of the PAN image and the spectral information of the MS image. However, most pansharpening methods fail to preserve spatial and spectral information simultaneously. In this article, we propose a framelet-based convolutional neural network (CNN) for pansharpening which makes it possible to pursue both high spectral and high spatial resolution. Our network consists of three subnetworks: 1) feature embedding net; 2) feature fusion net; and 3) framelet prediction net. Different from conventional CNN methods directly inferring high-resolution MS images, our approach learns to predict their framelet coefficients from available PAN and MS images. The introduction of multilevel feature aggregation and hybrid residual connection makes full use of spatial information of PAN image and spectral information of MS image. Quantitative and qualitative experiments at reduced- and full-resolution demonstrate that the proposed method achieves more appealing results than other state-of-the-art pansharpening methods. The source code and trained models are available at https://github.com/TingMAC/FrMLNet.

Coastal ocean areas are very dynamic regions subject to strong anthropogenic pressure (e.g. industry, tourism, renewable energies, population). Satellite data constitute a unique tool to study and monitor these complex areas at high spatial and temporal resolutions. While “traditional” ocean colour satellites like Sentinel-3 provide daily temporal resolution, their sensors do not measure at the spatial resolution needed to correctly resolve complex coastal dynamics. On the other hand, while high spatial resolution sensors, like the MSI onboard Sentinel-2 (10m - 60m resolution), are able to resolve these small scales, their revisit time is far from optimal (2-5 days for the Sentinel-2 A & B tandem). Additionally, both high spatial resolution datasets and traditional ones are hindered by the presence of clouds, resulting in a large amount of missing data. Given the high complementarity of these two measurement strategies (Sentinel-3 and Sentinel-2), we present a methodology to derive gap-filled multi resolution ocean colour products from the synergistic use of Sentinel-2 and Sentinel-3 data. Applying DINEOF (Data Interpolating Empirical Orthogonal Functions), we exploit and combine both the high-resolution spatial variability information contained in Sentinel-2 products as well as the high temporal information of Sentinel-3. Both Sentinel-2 and Sentinel-3 products are generated using the Copernicus Marine High-Resolution processor, which uses automated switching algorithms adapted to the local water conditions (at the pixel level) to retrieve optimal remote sensing spectra and water quality variables. With this approach, we address the high variability of different water types with small scale changes. The combined Sentinel-2 and Sentinel-3 products consist of Chlorophyll-a concentration and turbidity retrieved through a multi-algorithm approach with optimized quality flagging. An analysis of turbidity and chlorophyll daily data at 100m resolution in the northern Adriatic Sea and the Belgian coastal zone will be presented, alongside an analysis of the spatial scales resolved by the original and merged DINEOF datasets.

Hyperspectral (HS) images obtained from space are useful for monitoring different natural phenomena on regional to global scales. The Environmental Mapping and Analysis Program (EnMAP) is a satellite recently launched by Germany to monitor the environment and explore the capabilities of hyperspectral sensors in the 420 and 2450 nm range of the spectrum. However, the data captured by the EnMAP mission have a ground sampling distance (GSD) of 30 m. This limits the use of the data for some applications that require higher spatial resolution (<10 m). This study examines the potential for improving the resolution of hyperspectral data using high resolution multispectral (MS) data obtained by Cubesats. Specifically, this work uses the data captured by the PlanetScope constellation, which has more than 150 CubeSats in low Earth orbit, with a high spatial and temporal resolution. The approach adopted leverages (1) the spectral capability of the hyperspectral EnMAP sensor, with a bandwidth of 6.5 nm in the visible and near infrared (VNIR) range (420–1000 nm) and 10 nm in the SWIR range (900–2450 nm), and (2) the spatial capability of the multispectral PlanetScope data, with a GSD of 3 meters, to enable significant spatial improvements due to its high spatial resolution. The main components of this work include: (i) area of interest clipping (ii) data co-registration, (iii) HS-MS data fusion, and (iv) quality assessments using the Jointly Spectral and Spatial Quality Index (QNR). In this study, a 2 km x 2 km area of interest was selected in the Malaucene region of France, where six state-of-the-art HS-MS fusion methods were evaluated: (1) fast multi-band image fusion algorithm (FUSE), (2) coupled nonnegative matrix factorization (CNMF), (3) smoothing filtered-based intensity modulation (SFIMHS), (4) maximum a posteriori stochastic mixing model (MAPSMM), (5) Hyperspectral Superresolution (HySure), and (6) generalized laplacian pyramid hypersharpening (GLPHS). Quality assessments of the enhanced data showed that high spectral and spatial fidelity are maintained, with the best performing fusion method being FUSE with a QNR of 0.625 followed by the MAPSMM method with a QNR of 0.604. Overall, this study advocates the benefits associated with the fusion of hyperspectral and multispectral data to obtain enhanced EnMAP data at 3 m GSD. 

Optical microscopes are essential equipment for observing and analysing cultured cells. In observations using a normal optical microscope, as the contrast of cells is rather low, phase contrast and differential interference microscopes are commonly used. In some cases, specific proteins in cells are fluorescently labelled and observed using a fluorescence microscope. In such observations, cells are generally cultured on a glass surface and observed through the glass. Therefore, spherical aberration occurs due to the refraction of light on the glass surface. This spherical aberration becomes larger when using an objective lens with higher magnification and higher numerical aperture, resulting in a decrease of spatial resolution and contrast. Here, we found that by observing cells cultured on an extremely thin silicon nitride (SiN) film (50 nm thick) under a conventional optical microscope, we could directly observe the cells at high resolution without spherical aberration. This improvement of spatial resolution was confirmed with both inverted and upright optical microscopes. Using a normal 100 × objective lens without oil immersion, we were able to directly observe melanosomes in melanoma cells and analyse their dynamic movement. We believe that using our original sample holder with thin SiN film will make it easy to observe live cells at high resolution and that this method will contribute to a wide range of biological research.

Particle image velocimetry (PIV) techniques have a limited field of view of the flow field and can only capture high-resolution flow fields in localized areas. To obtain a larger measurement range, multiple cameras must be used to capture the flow field simultaneously and then stitch the parts together. However, this method can be expensive. We propose the local-global fusion convolutional neural network (LGF-CNN) for reconstructing large-field flow fields with high spatial resolution based only on two flow data types: local small-field high spatial resolution wake velocity fields and global large-field low spatial resolution velocity fields. The core of the model consists of convolutional neural network (CNN) architecture to learn the mapping relationship between the small field of view with high spatial resolution and the large field of view with low spatial resolution. Using the effectively trained LGF-CNN model, we demonstrate its ability to reconstruct high-resolution velocity fields around the circular cylinder. The LGF-CNN is rigorously validated on a number of representative datasets, including simulated data for Reynolds numbers of 200 and 500, as well as experimental data for a Reynolds number of 3.3 × 104 with the steady jet at the rear stagnation point of the cylinder. The results demonstrate the ability of LGF-CNN to generate accurate velocity fields with high spatial resolution, including reliable detection of high-frequency components. The proposed method could reduce the number of cameras required for large-field, high spatial resolution PIV measurements, thereby reducing experimental costs.

Comparison of visible and multi-satellite global inundation datasets at high-spatial resolution

MODIS and ASTER synergy for characterizing thermal volcanic activity

Hyperspectral remote sensing images (HSIs) are characterized by having a low spatial resolution and a high spectral resolution, whereas multispectral images (MSIs) are characterized by low spectral and high spatial resolutions. These complementary characteristics have stimulated active research in the inference of images with high spatial and spectral resolutions from HSI-MSI pairs. In this paper, we formulate this data fusion problem as the minimization of a convex objective function containing two data-fitting terms and an edge-preserving regularizer. The data-fitting terms are quadratic and account for blur, different spatial resolutions, and additive noise; the regularizer, a form of vector Total Variation, promotes aligned discontinuities across the reconstructed hyperspectral bands. The optimization described above is rather hard, owing to its non-diagonalizable linear operators, to the non-quadratic and non-smooth nature of the regularizer, and to the very large size of the image to be inferred. We tackle these difficulties by tailoring the Split Augmented Lagrangian Shrinkage Algorithm (SALSA) — an instance of the Alternating Direction Method of Multipliers (ADMM) — to this optimization problem. By using a convenient variable splitting and by exploiting the fact that HSIs generally “live” in a low-dimensional subspace, we obtain an effective algorithm that yields state-of-the-art results, as illustrated by experiments.

Momentum‐resolved electron energy loss spectroscopy (MREELS) probes the momentum (q) dependence (dispersion) of energy losses from characteristic excitations such as excitons, plasmon, and interband excitations thus probing bandstructures, indirect excitations, and dipole‐forbidden excitations. 1 However, the ability to obtain momentum‐resolved spectra both at high spatial and momentum resolutions has remained difficult. As such, layer‐resolved momentum spectra from ultra‐thin films, heterostructures as well as from nanostructures has not been widely reported. We present an experimental approach that enables the acquisition of momentum resolved spectra at high spatial resolution down to 2 nm using a nano‐beam electron diffraction approach. 2 Through this approach we have obtained momentum resolved spectra from individual, differently‐oriented nano‐domains in an ultra thin (12 nm) PrNiO 3 layer as well as spectra from different positions in a LaNiO 3 thin film (70 nm). Figure 1(a) displays a Z‐contrast image of the LaNiO 3 thin layer grown on a LaSrAlO 4 substrate. A nano‐beam electron diffraction pattern from the LaNiO 3 layer is shown in Figure 1(b). A slit is used to select the 200, ‐200 set of diffraction spots (shown by the red rectangle) allowing electrons that have been scattered to certain scattering angles into the spectrometer. This corresponds to the Г‐X direction of the Brillouin zone in the pseudo‐cubic symmetry of LaNiO 3 . The resulting image (ω‐q map) shown in figure 1(c) displays energy losses as function of momentum transfer along the Г‐X (q// [100] ) direction in the reciprocal space. The presented approach will enable the acquisition of momentum resolved spectra from nano‐structured materials, thin films, interfaces, surfaces, and heterostructures at high spatial, energy, and momentum resolutions.

The regulated release of neurotransmitter occurs via the fusion of synaptic vesicles (SVs) at specialized regions of the presynaptic membrane called active zones (AZs). These regions are defined by a cytoskeletal matrix assembled at AZs (CAZ), which functions to direct SVs toward docking and fusion sites and supports their maturation into the readily releasable pool. In addition, CAZ proteins localize voltage-gated Ca(2+) channels at SV release sites, bringing the fusion machinery in close proximity to the calcium source. Proteins of the CAZ therefore ensure that vesicle fusion is temporally and spatially organized, allowing for the precise and reliable release of neurotransmitter. Importantly, AZs are highly dynamic structures, supporting presynaptic remodeling, changes in neurotransmitter release efficacy, and thus presynaptic forms of plasticity. In this review, we discuss recent advances in the study of active zones, highlighting how the CAZ molecularly defines sites of neurotransmitter release, endocytic zones, and the integrity of synapses.

Detecting damage in structures based on the change in their dynamics or modal parameters (modal frequencies and mode shapes) has been extensively studied for three decades. The success of such a global, passive, vibration-based method in field applications, however, has long been hindered by the bottleneck of low spatial resolution vibration sensor measurements. The primary reason is that damage typically initiates and develops in local regions that need to be captured and characterized by very high spatial resolution vibration measurements and modal parameters (mode shapes), which are extremely difficult to obtain using traditional vibration measurement techniques. For example, accelerometers and strain-gauge sensors are typically placed at a limited number of discrete locations, providing low spatial resolution vibration measurements. Laser vibrometers provide high-resolution measurements, but are expensive and make sequential measurements that are time- and labor-consuming. Recently, digital video cameras—which are relatively low cost, agile, and able to provide high spatial resolution, simultaneous, pixel measurements—have emerged as a promising tool to achieve full-field, high spatial resolution vibration measurements. Combined with advanced vision processing and unsupervised machine algorithms, a new method has recently been developed to blindly and efficiently extract the full-field, high-resolution, dynamic parameters from the video measurements of an operating, output-only structure. This work studies the feasibility of performing damage detection using the full-field, very high spatial resolution mode shape (of the fundamental mode) blindly extracted from the video of the operating (output-only) structure without any knowledge of reference (healthy) structural information. A spatial fractal dimension analysis is applied on the full-field mode shape of the damaged structure to detect damage-induced irregularity. Additionally, the equivalence between the fractal dimension and the squared curvature (modal strain energy) of the mode shape curve, when of high spatial resolution, is mathematically derived. Laboratory experiments are conducted on bench-scale structures, including a building structure and a cantilever beam, to validate the approach. The results illustrate that using the full-field, very high-resolution mode shape enables detection of minute, non-visible, damage in a global, completely passive sensing manner, which was previously not possible to achieve.