Recent Progress in the Correlative Structured Illumination Microscopy

Structured illumination microscopy (SIM) is one of the most significant widefield super-resolution optical imaging techniques. The conventional SIM utilizes a sinusoidal structured pattern to excite the fluorescent sample; which eventually down-modulates higher spatial frequency sample information within the diffraction-limited passband of the microscopy system and provides around two-fold resolution enhancement over diffraction limit after suitable computational post-processing. Here we provide an overview of the basic principle, image reconstruction, technical development of the SIM technique. Nonetheless, in order to push the SIM resolution further towards the extreme nanoscale dimensions, several different approaches are launched apart from the conventional SIM. Among the various SIM methods, some of the important techniques e.g. TIRF, non-linear, plasmonic, speckle SIM etc are discussed elaborately. Moreover, we highlight different implementations of SIM in various other imaging modalities to enhance their imaging performances with augmented capabilities. Finally, some future outlooks are mentioned which might develop fruitfully and pave the way for new discoveries in near future.

Structured illumination microscopy (SIM) is a popular and powerful super-resolution (SR) technique in biomedical research. However, the conventional reconstruction algorithm for SIM heavily relies on the accurate prior knowledge of illumination patterns and signal-to-noise ratio (SNR) of raw images. To obtain high-quality SR images, several raw images need to be captured under high fluorescence level, which further restricts SIM’s temporal resolution and its applications. Deep learning (DL) is a data-driven technology that has been used to expand the limits of optical microscopy. In this study, we propose a deep neural network based on multi-level wavelet and attention mechanism (MWAM) for SIM. Our results show that the MWAM network can extract high-frequency information contained in SIM raw images and accurately integrate it into the output image, resulting in superior SR images compared to those generated using wide-field images as input data. We also demonstrate that the number of SIM raw images can be reduced to three, with one image in each illumination orientation, to achieve the optimal tradeoff between temporal and spatial resolution. Furthermore, our MWAM network exhibits superior reconstruction ability on low-SNR images compared to conventional SIM algorithms. We have also analyzed the adaptability of this network on other biological samples and successfully applied the pretrained model to other SIM systems.

Structured illumination microscopy (SIM) is one of the most powerful and versatile super-resolution methods due to its live-cell imaging ability of subcellular structures with high speed. In this paper, we propose an alternative SIM assisted by Bloch surface wave (BSW). Through replacing the conventional laser interference fringes with sub-diffraction BSW counterparts with higher spatial frequency, the imaging resolution of SIM could be enhanced greatly and the super-resolution imaging capability down to 80 nm could be achieved. Compared with traditional wide-field fluorescence microscopy, this BSW SIM demonstrates 2.78 times enhancement in imaging resolution, which surpasses general SIM with only 2 times improvement. Moreover, the structured illumination intensity could be boosted drastically, which is beneficial to nonlinear super-resolution imaging techniques, such as saturated SIM. This BSW SIM would provide a super-resolution, wide field of view approach for surface super-resolution fluorescent imaging while maintaining high imaging speed and good bio-compatibility.

Structured illumination microscopy (SIM) is a fast and gentle super-resolution fluorescence imaging technique, featuring live-cell compatible excitation light levels and high imaging speeds. To achieve SIM, spatial modulation of the fluorescence excitation light is employed. This is typically achieved by interfering coherent laser beams in the sample plane, which are often created by spatial light modulators (SLMs). Digital micromirror devices (DMDs) are a form of SLMs with certain advantages, such as high speed, low cost and wide availability, which present certain hurdles in their implementation, mainly the blazed grating effect caused by the jagged surface structure of the tilted mirrors. Recent works have studied this effect through modelling, simulations and experiments, and laid out possible implementations of multi-color SIM imaging based on DMDs. Here, we present an implementation of a dual-color DMD based SIM microscope using temperature-controlled wavelength matching. By carefully controlling the output wavelength of a diode laser by temperature, we can tune two laser wavelengths in such a way that no opto-mechanical realignment of the SIM setup is necessary when switching between both wavelengths. This reduces system complexity and increases imaging speed. With measurements on nano-bead reference samples, as well as the actin skeleton and membrane of fixed U2OS cells, we demonstrate the capabilities of the setup.

Structured illumination microscopy (SIM) is an important super-resolution-based microscopy technique that breaks the diffraction limit and enhances optical microscopy systems. With the development of biology and medical engineering, there is a high demand for real-time and robust SIM imaging under extreme low-light and short-exposure environments. Existing SIM techniques typically require multiple structured illumination frames to produce a high-resolution image. In this article, we propose a single-frame SIM (SF-SIM) based on deep learning. Our SF-SIM only needs one shot of a structured illumination frame and generates similar results compared with the traditional SIM systems that typically require 15 shots. In our SF-SIM, we propose a noise estimator that can effectively suppress the noise in the image and enable our method to work in the low-light and short-exposure environment without the need for stacking multiple frames for nonlocal denoising. We also design a bandpass attention module that makes our deep network more sensitive to the change of frequency and enhances the imaging quality. Our proposed SF-SIM is almost 14 times faster than traditional SIM methods when achieving similar results. Therefore, our method is significantly valuable for the development of microbiology and medicine.

Structured illumination microscopy (SIM) has emerged as a promising super-resolution fluorescence imaging technique, offering diverse configurations and computational strategies to mitigate phototoxicity during real-time imaging of biological specimens. Traditional efforts to enhance system frame rates have concentrated on processing algorithms, like rolling reconstruction or reduced frame reconstruction, or on investments in costly sCMOS cameras with accelerated row readout rates. In this article, we introduce an approach to elevate SIM frame rates and region of interest (ROI) coverage at the hardware level, without necessitating an upsurge in camera expenses or intricate algorithms. Here, parallel acquisition-readout SIM (PAR-SIM) achieves the highest imaging speed for fluorescence imaging at currently available detector sensitivity. By using the full frame-width of the detector through synchronizing the pattern generation and image exposure-readout process, we have achieved a fundamentally stupendous information spatial-temporal flux of 132.9 MPixels · s−1, 9.6-fold that of the latest techniques, with the lowest SNR of −2.11 dB and 100 nm resolution. PAR-SIM demonstrates its proficiency in successfully reconstructing diverse cellular organelles in dual excitations, even under conditions of low signal due to ultra-short exposure times. Notably, mitochondrial dynamic tubulation and ongoing membrane fusion processes have been captured in live COS-7 cell, recorded with PAR-SIM at an impressive 408 Hz. We posit that this novel parallel exposure-readout mode not only augments SIM pattern modulation for superior frame rates but also holds the potential to benefit other complex imaging systems with a strategic controlling approach.

The objective of optical super-resolution imaging is to acquire reliable sub-diffraction information on bioprocesses to facilitate scientific discovery. Structured illumination microscopy (SIM) is acknowledged as the optimal modality for live-cell super-resolution imaging. Although recent deep learning techniques have substantially advanced SIM, their transparency and reliability remain uncertain and under-explored, often resulting in unreliable results and biological misinterpretation. Here, we develop Bayesian deep learning (BayesDL) for SIM, which enhances the reconstruction of densely labeled structures while enabling the quantification of super-resolution uncertainty. With the uncertainty, BayesDL-SIM achieves high-fidelity distribution-informed SIM imaging, allowing for the communication of credibility estimates to users regarding the model outcomes. We also demonstrate that BayesDL-SIM boosts SIM reliability by identifying and preventing erroneous generalizations in various model misuse scenarios. Moreover, the BayesDL uncertainty shows versatile utilities for daily super-resolution imaging, such as error estimation, data acquisition evaluation, etc. Furthermore, we demonstrate the effectiveness and superiority of BayesDL-SIM in live-cell imaging, which reliably reveals F-actin dynamics and the reorganization of the cell cytoskeleton. This work lays the foundation for the reliable implementation of deep learning-based SIM methods in practical applications.

Structured illumination microscopy (SIM) is an attractive choice for fast super-resolution imaging. The generation of structured illumination patterns made by interference of laser beams is broadly employed to obtain high modulation depth of patterns, while the polarizations of the laser beams must be elaborately controlled to guarantee the high contrast of interference intensity, which brings a more complex configuration for the polarization control. The emerging pattern projection strategy is much more compact, but the modulation depth of patterns is deteriorated by the optical transfer function (OTF) of the optical system, especially in high spatial frequency near diffraction limit. Therefore, the traditional super-resolution reconstruction algorithm for interference-based SIM will suffer from much artifact in the case of projection-based SIM that possesses a low modulation depth. Here, we propose an alternative reconstruction algorithm based on image recombination transform (IRT), which provides an alternative solution to address this problem even in a weak modulation depth. We demonstrated the effectiveness of this algorithm in the multicolor super-resolution imaging of BPAE cells in our developed projection-based SIM system, which applies a computer controlled digital micro-mirror device (DMD) for fast fringe generation and multicolor LEDs for illumination. The merit of the system incorporated with the proposed algorithm allows for a low excitation intensity fluorescence imaging even less than 1W/cm2, which is beneficial for the long-term, in vivo super-resolved imaging of live cells and tissues.

μLinear Structured Illumination Microscopy (SIM) provides a two-fold increase over the diffraction limited resolution. SIM produces excellent images with 120nm resolution in tissue culture cells in two and three dimensions. For SIM to work correctly, the point spread function (PSF) and optical transfer function (OTF) must be known, and, ideally, should be unaberrated. When imaging through thick samples, aberrations will be introduced into the optical system which will reduce the peak intensity and increase the width of the PSF. This will lead to reduced resolution and artifacts in SIM images. Adaptive optics can be used to correct the optical wavefront restoring the PSF to its unaberrated state, and AO has been used in several types of fluorescence microscopy. We demonstrate that AO can be used with SIM to achieve 120nm resolution through 25m of tissue by imaging through the full thickness of an adult C. elegans roundworm. The aberrations can be corrected over a 25μm × 45μm field of view with one wavefront correction setting, demonstrating that AO can be used effectively with widefield superresolution techniques.

We propose a super resolution imaging technique called plasmonic structured illumination microscopy (PSIM), which combines the structured illumination microscopy technique with the tunable surface plasmon interference. Because of the high-resolution enabled by using surface plasmon interference as an illumination source, PSIM possesses higher image resolving power compared with conventional structured illumination microscopy. To demonstrate the technique, we present two specific types of plasmonic structure designs for PSIM. The final images from the simulations show 3-fold and 4-fold resolution improvement compared with conventional epi-fluorescence microscopy.

Structured illumination microscopy (SIM) is a widely adopted super-resolution imaging technique. Conventional 3D-SIM requires at least 15 exposures at each z-plane to achieve ∼2 × improved lateral and axial resolution. However, this requirement for a large number of exposures for “super-resolution” exacerbates photobleaching and slows imaging speed, thus significantly limiting its application in volumetric biological imaging. Here, we introduce multi-scan SIM (MS-SIM) that integrates a simple beam splitter for simultaneously imaging three different focal planes and a deformable mirror that enables rapid z-scanning over three contiguous sub-volumes. We demonstrate the MS-SIM system through high-quality live whole cell SIM imaging at ∼1 Hz. The high efficiency and flexibility of MS-SIM can significantly impact 3D super-resolution imaging of biological and dense colloidal systems.

Structured-illumination microscopy (SIM) offers a twofold resolution enhancement beyond the optical diffraction limit. At present, SIM requires several raw structured-illumination (SI) frames to reconstruct a super-resolution (SR) image, especially the time-consuming reconstruction of speckle SIM, which requires hundreds of SI frames. Considering this, we herein propose an untrained structured-illumination reconstruction neural network (USRNN) with known illumination patterns to reduce the amount of raw data that is required for speckle SIM reconstruction by 20 times and thus improve its temporal resolution. Benefiting from the unsupervised optimizing strategy and CNNs' structure priors, the high-frequency information is obtained from the network without the requirement of datasets; as a result, a high-fidelity SR image with approximately twofold resolution enhancement can be reconstructed using five frames or less. Experiments on reconstructing non-biological and biological samples demonstrate the high-speed and high-universality capabilities of our method.

Structured illumination microscopy (SIM) allows non-invasive visualization of nanoscale subcellular structures. However, image acquisition and reconstruction become the bottleneck to further improve the imaging speed. Here, we propose a method to accelerate SIM imaging by combining the spatial re-modulation principle with Fourier domain filtering and using measured illumination patterns. This approach enables high-speed, high-quality imaging of dense subcellular structures using a conventional nine-frame SIM modality without phase estimation of the patterns. In addition, seven-frame SIM reconstruction and additional hardware acceleration further improve the imaging speed using our method. Furthermore, our method is also applicable to other spatially uncorrelated illumination patterns, such as distorted sinusoidal, multifocal, and speckle patterns.

Nonlinear structured illumination microscopy (SIM) allows full-field imaging at resolutions <100 nm. Two nonlinear effects, excitation saturation (SSIM) and the photo-switching of protein had been applied to nonlinear SIM. We report a new SIM technique which utilizes the nonlinearity of STED effect. Resolution and signal noise ratio simulation shows that STED-SIM may serve as a better alternative to SSIM and SIM with photo-switchable protein. SIM requires a strong nonlinear effect in a large area. We use Surface Plasmon Resonant to enhance of evanescence field near a dielectric-metal-dielectric interface. An 8 times STED effect enhancement is achieved on an optimized glass-silver-glass-water planar structure. We further use the interference of two SPR-enhanced STED fields propagating at opposite direction to generate a 1D structured STED field. Combined with a uniform excitation field, the structure STED field allows full field total internal reflection imaging with an enhanced resolution along the structured dimension. Less than 50 nm resolution is demonstrated. A STED-SIM microscope with 2D structured STED field is under development. Future research will apply the microscope to superresolution imaging of membrane resident or near membrane structure at super-resolution in live cells.

Structured illumination microscopy (SIM) is a super-resolution technique that uses patterned excitation light to achieve an optical resolution that is twice the diffraction limit. It does not require special buffers or dyes. In this chapter, we discuss the implementation of structured illumination in fluorescence microscopy, show how to perform SIM acquisition and processing, and discuss typical SIM acquisition and processing artefacts and how to address them.