Bayesian deep-learning structured illumination microscopy enables reliable super-resolution imaging with uncertainty quantification

Structured illumination microscopy (SIM) is one of the most powerful and versatile super-resolution methods due to its live-cell imaging ability of subcellular structures with high speed. In this paper, we propose an alternative SIM assisted by Bloch surface wave (BSW). Through replacing the conventional laser interference fringes with sub-diffraction BSW counterparts with higher spatial frequency, the imaging resolution of SIM could be enhanced greatly and the super-resolution imaging capability down to 80 nm could be achieved. Compared with traditional wide-field fluorescence microscopy, this BSW SIM demonstrates 2.78 times enhancement in imaging resolution, which surpasses general SIM with only 2 times improvement. Moreover, the structured illumination intensity could be boosted drastically, which is beneficial to nonlinear super-resolution imaging techniques, such as saturated SIM. This BSW SIM would provide a super-resolution, wide field of view approach for surface super-resolution fluorescent imaging while maintaining high imaging speed and good bio-compatibility.

Structured illumination microscopy (SIM) is a popular and powerful super-resolution (SR) technique in biomedical research. However, the conventional reconstruction algorithm for SIM heavily relies on the accurate prior knowledge of illumination patterns and signal-to-noise ratio (SNR) of raw images. To obtain high-quality SR images, several raw images need to be captured under high fluorescence level, which further restricts SIM’s temporal resolution and its applications. Deep learning (DL) is a data-driven technology that has been used to expand the limits of optical microscopy. In this study, we propose a deep neural network based on multi-level wavelet and attention mechanism (MWAM) for SIM. Our results show that the MWAM network can extract high-frequency information contained in SIM raw images and accurately integrate it into the output image, resulting in superior SR images compared to those generated using wide-field images as input data. We also demonstrate that the number of SIM raw images can be reduced to three, with one image in each illumination orientation, to achieve the optimal tradeoff between temporal and spatial resolution. Furthermore, our MWAM network exhibits superior reconstruction ability on low-SNR images compared to conventional SIM algorithms. We have also analyzed the adaptability of this network on other biological samples and successfully applied the pretrained model to other SIM systems.

Most current superresolution (SR) microscope techniques surpass the diffraction limit at the expense of temporal resolution, compromising their applications to live-cell imaging. Here we describe a new SR fluorescence microscope based on confocal microscope optics, which we name the spinning disk superresolution microscope (SDSRM). Theoretically, the SDSRM is equivalent to a structured illumination microscope (SIM) and achieves a spatial resolution of 120 nm, double that of the diffraction limit of wide-field fluorescence microscopy. However, the SDSRM is 10 times faster than a conventional SIM because SR signals are recovered by optical demodulation through the stripe pattern of the disk. Therefore a single SR image requires only a single averaged image through the rotating disk. On the basis of this theory, we modified a commercial spinning disk confocal microscope. The improved resolution around 120 nm was confirmed with biological samples. The rapid dynamics of micro-tubules, mitochondria, lysosomes, and endosomes were observed with temporal resolutions of 30-100 frames/s. Because our method requires only small optical modifications, it will enable an easy upgrade from an existing spinning disk confocal to a SR microscope for live-cell imaging.

Structured illumination microscopy (SIM) is an attractive choice for fast super-resolution imaging. The generation of structured illumination patterns made by interference of laser beams is broadly employed to obtain high modulation depth of patterns, while the polarizations of the laser beams must be elaborately controlled to guarantee the high contrast of interference intensity, which brings a more complex configuration for the polarization control. The emerging pattern projection strategy is much more compact, but the modulation depth of patterns is deteriorated by the optical transfer function (OTF) of the optical system, especially in high spatial frequency near diffraction limit. Therefore, the traditional super-resolution reconstruction algorithm for interference-based SIM will suffer from much artifact in the case of projection-based SIM that possesses a low modulation depth. Here, we propose an alternative reconstruction algorithm based on image recombination transform (IRT), which provides an alternative solution to address this problem even in a weak modulation depth. We demonstrated the effectiveness of this algorithm in the multicolor super-resolution imaging of BPAE cells in our developed projection-based SIM system, which applies a computer controlled digital micro-mirror device (DMD) for fast fringe generation and multicolor LEDs for illumination. The merit of the system incorporated with the proposed algorithm allows for a low excitation intensity fluorescence imaging even less than 1W/cm2, which is beneficial for the long-term, in vivo super-resolved imaging of live cells and tissues.

Proper membrane localization of ion channels is essential for the function of neuronal cells. Particularly, the computational ability of dendrites depends on the localization of different ion channels in specific subcompartments. However, the molecular mechanisms that control ion channel localization in distinct dendritic subcompartments are largely unknown. Here, we developed a quantitative live cell imaging method to analyze protein sorting and post-Golgi vesicular trafficking. We focused on two dendritic voltage-gated potassium channels that exhibit distinct localizations: Kv2.1 in proximal dendrites and Kv4.2 in distal dendrites. Our results show that Kv2.1 and Kv4.2 channels are sorted into two distinct populations of vesicles at the Golgi apparatus. The targeting of Kv2.1 and Kv4.2 vesicles occurred by distinct mechanisms as evidenced by their requirement for specific peptide motifs, cytoskeletal elements, and motor proteins. By live cell and super-resolution imaging, we identified a novel trafficking machinery important for the localization of Kv2.1 channels. Particularly, we identified non-muscle myosin II as an important factor in Kv2.1 trafficking. These findings reveal that the sorting of ion channels at the Golgi apparatus and their subsequent trafficking by unique molecular mechanisms are crucial for their specific localizations within dendrites.

Structured illumination microscopy (SIM) is one of the most significant widefield super-resolution optical imaging techniques. The conventional SIM utilizes a sinusoidal structured pattern to excite the fluorescent sample; which eventually down-modulates higher spatial frequency sample information within the diffraction-limited passband of the microscopy system and provides around two-fold resolution enhancement over diffraction limit after suitable computational post-processing. Here we provide an overview of the basic principle, image reconstruction, technical development of the SIM technique. Nonetheless, in order to push the SIM resolution further towards the extreme nanoscale dimensions, several different approaches are launched apart from the conventional SIM. Among the various SIM methods, some of the important techniques e.g. TIRF, non-linear, plasmonic, speckle SIM etc are discussed elaborately. Moreover, we highlight different implementations of SIM in various other imaging modalities to enhance their imaging performances with augmented capabilities. Finally, some future outlooks are mentioned which might develop fruitfully and pave the way for new discoveries in near future.

For almost a century, the resolution of optical microscopy was thought to be limited by Abbé's law describing the diffraction limit of light. At the turn of the millennium, aided by new technologies and fluorophores, the field of optical microscopy finally surpassed the diffraction barrier: a milestone achievement that has been recognized by the 2014 Nobel Prize in Chemistry. Many super-resolution methods rely on the unique photophysical properties of the fluorophores to improve resolution, posing significant limitations on biological imaging, such as multicoloured staining, live-cell imaging and imaging thick specimens. Structured Illumination Microscopy (SIM) is one branch of super-resolution microscopy that requires no such special properties of the applied fluorophores, making it more versatile than other techniques. Since its introduction in biological imaging, SIM has proven to be a popular tool in the biologist's arsenal for following biological interaction and probing structures of nanometre scale. SIM continues to see much advancement in design and implementation, including the development of Image Scanning Microscopy (ISM), which uses patterned excitation via either predefined arrays or raster-scanned single point-spread functions (PSF). This review aims to give a brief overview of the SIM and ISM processes and subsequent developments in the image reconstruction process. Drawing from this, and incorporating more recent achievements in light shaping (i.e. pattern scanning and super-resolution beam shaping), this study also intends to suggest potential future directions for this ever-expanding field.

Super-resolution structured illumination microscopy (SR-SIM) is one of the important techniques that are most suitable for live-cell imaging. The reconstructed SR-SIM images are noisy once the raw images are recorded with low-light exposure. Here, we propose a new network (entitled the ND-SIM network) to denoise the SR images reconstructed using frequency-domain algorithms (FDAs). We demonstrate that ND-SIM can yield artifact-free SR images using raw images with an average photon count down to 20 per pixel while achieving comparable resolution to the ground truth (GT) obtained with high-light exposure. We can envisage that the ND-SIM will be widely applied for the long-term, super-resolution live-cell imaging of various bioprocesses in the future.

Structured illumination microscopy (SIM) is a fast and gentle super-resolution fluorescence imaging technique, featuring live-cell compatible excitation light levels and high imaging speeds. To achieve SIM, spatial modulation of the fluorescence excitation light is employed. This is typically achieved by interfering coherent laser beams in the sample plane, which are often created by spatial light modulators (SLMs). Digital micromirror devices (DMDs) are a form of SLMs with certain advantages, such as high speed, low cost and wide availability, which present certain hurdles in their implementation, mainly the blazed grating effect caused by the jagged surface structure of the tilted mirrors. Recent works have studied this effect through modelling, simulations and experiments, and laid out possible implementations of multi-color SIM imaging based on DMDs. Here, we present an implementation of a dual-color DMD based SIM microscope using temperature-controlled wavelength matching. By carefully controlling the output wavelength of a diode laser by temperature, we can tune two laser wavelengths in such a way that no opto-mechanical realignment of the SIM setup is necessary when switching between both wavelengths. This reduces system complexity and increases imaging speed. With measurements on nano-bead reference samples, as well as the actin skeleton and membrane of fixed U2OS cells, we demonstrate the capabilities of the setup.

Structured illumination microscopy (SIM) is a widely adopted super-resolution imaging technique. Conventional 3D-SIM requires at least 15 exposures at each z-plane to achieve ∼2 × improved lateral and axial resolution. However, this requirement for a large number of exposures for “super-resolution” exacerbates photobleaching and slows imaging speed, thus significantly limiting its application in volumetric biological imaging. Here, we introduce multi-scan SIM (MS-SIM) that integrates a simple beam splitter for simultaneously imaging three different focal planes and a deformable mirror that enables rapid z-scanning over three contiguous sub-volumes. We demonstrate the MS-SIM system through high-quality live whole cell SIM imaging at ∼1 Hz. The high efficiency and flexibility of MS-SIM can significantly impact 3D super-resolution imaging of biological and dense colloidal systems.

Genetic code expansion and bioorthogonal labeling provide for the first time a way for direct, site-specific labeling of proteins with fluorescent-dyes in live cells. Although the small size and superb photophysical parameters of fluorescent-dyes offer unique advantages for high-resolution microscopy, this approach has yet to be embraced as a tool in live cell imaging. Here we evaluated the feasibility of this approach by applying it for α-tubulin labeling. After a series of calibrations, we site-specifically labeled α-tubulin with silicon rhodamine (SiR) in live mammalian cells in an efficient and robust manner. SiR-labeled tubulin successfully incorporated into endogenous microtubules at high density, enabling video recording of microtubule dynamics in interphase and mitotic cells. Applying this labeling approach to structured illumination microscopy resulted in an increase in resolution, highlighting the advantages in using a smaller, brighter tag. Therefore, using our optimized assay, genetic code expansion provides an attractive tool for labeling proteins with a minimal, bright tag in quantitative high-resolution imaging.

Abstract: As an early component of modern super-resolution (SR) imaging technology, structured illumination microscopy (SIM) has been developed for nearly twenty years. With up to ~60 nm and 564 Hz frame rate, it has achieved an optimal combination of spatiotemporal resolution in live cells recently. Despite these advantages, SIM also suffers disadvantages, some of which originated from the intrinsic reconstruction process. Here we review recent technical advances in SIM, including SR reconstruction, performance evaluation, and its integration with other technologies to provide a practical guide for biologists.

Structured-illumination microscopy (SIM) offers a twofold resolution enhancement beyond the optical diffraction limit. At present, SIM requires several raw structured-illumination (SI) frames to reconstruct a super-resolution (SR) image, especially the time-consuming reconstruction of speckle SIM, which requires hundreds of SI frames. Considering this, we herein propose an untrained structured-illumination reconstruction neural network (USRNN) with known illumination patterns to reduce the amount of raw data that is required for speckle SIM reconstruction by 20 times and thus improve its temporal resolution. Benefiting from the unsupervised optimizing strategy and CNNs' structure priors, the high-frequency information is obtained from the network without the requirement of datasets; as a result, a high-fidelity SR image with approximately twofold resolution enhancement can be reconstructed using five frames or less. Experiments on reconstructing non-biological and biological samples demonstrate the high-speed and high-universality capabilities of our method.

Chapter 16 - Seeing more with structured illumination microscopy

Structured illumination microscopy (SIM) achieves superresolution in fluorescence imaging through patterned illumination and computational image reconstruction, yet current methods require bulky, costly modulation optics and high-precision optical alignment, thus hindering the widespread implementation of SIM. To address this challenge, this work demonstrates how nano-optical metasurfaces, rationally designed to tailor the far-field optical wavefront at sub-wavelength dimensions, hold great potential as ultrathin, single-surface, all-optical wavefront modulators for SIM. We computationally demonstrate this principle with a multipolar-resonant metasurface composed of silicon nanostructures that generate versatile optical wavefronts in the far field upon variation of the polarization or angle of incident light. Algorithmic optimization is performed to identify the seven most suitable illumination patterns for SIM generated by the metasurface based on three key criteria. We quantitatively demonstrate that multipolar-resonant metasurface SIM (mrm-SIM) achieves resolution gain that is comparable to conventional methods by applying the seven optimal metasurface-generated wavefronts to simulated fluorescent objects and reconstructing the objects using proximal gradient descent. Notably, we show that mrm-SIM achieves these resolution gains with a far-field illumination pattern that circumvents complex equipment and alignment requirements of comparable methodologies. The work presented here paves the way for a metasurface-enabled experimental simplification of structured illumination microscopy.