Digital-SMLM for precisely localizing emitters within the diffraction limit.

Caveolae are plasma membrane invaginations whose formation requires caveolin-1 (Cav1), the adaptor protein polymerase I, and the transcript release factor (PTRF or CAVIN1). Caveolae have an important role in cell functioning, signaling, and disease. In the absence of CAVIN1/PTRF, Cav1 forms non-caveolar membrane domains called scaffolds. In this work, we train machine learning models to automatically distinguish between caveolae and scaffolds from single molecule localization microscopy (SMLM) data. We apply machine and deep learning algorithms to discriminate biological structures from SMLM data. Our work is the first that is leveraging machine and deep learning approaches to automatically identifying biological structures from SMLM data. In particular, we develop and compare three binary classification methods to identify whether or not a given 3D cluster of Cav1 proteins is a caveolae. The first uses a random forest classifier applied to 28 hand-picked features, the second uses a convolutional neural net (CNN) applied to a projection of the point clouds onto three planes, and the third uses a PointNet model, a recent development that can directly take point clouds as its input. We validate our methods on a dataset of super-resolution microscopy images of PC3 prostate cancer cells labeled for Cav1. Specifically, we have images from two cell populations: 10 PC3 and 10 CAVIN1/PTRF-transfected PC3 cells (PC3-PTRF cells) that form caveolae. We obtained a balanced set of 1714 different cellular structures. Our results show that both the random forest on hand selected features and the deep learning approach achieve high accuracy in distinguishing the intrinsic features of the caveolae and non-caveolae biological structures. More specifically, both random forest and deep CNN classifiers achieve classification accuracy reaching 94% on our test set, while the PointNet model only reached 83% accuracy. We also discuss the pros and cons of the different approaches. Additional information about individual files is in the accompanying CSV file (item_metadata.csv). This dataset was originally deposited in the Simon Fraser University institutional repository.

The „Resolution Revolution in fluorescence microscopy over the last decade has given rise to a variety of techniques that allow imaging beyond the diffraction limit with a resolution power down into the nanometer range. With this, the field of so-called super-resolution microscopy was born. It allows to visualize cellular architecture at a molecular level and thereby achieve a resolution level that had been previously only accessible by electron microscopy approaches. One of these promising techniques is single molecule localization microscopy (SMLM) in its most varied forms such as direct stochastic optical reconstruction microscopy (dSTORM) which are based on the temporal separation of the emission of individual fluorophores. Localization analysis of the subsequently taken images of single emitters eventually allows to reconstruct an image containing super-resolution information down to typically 20 nm in a cellular setting. The key point here is the localization precision, which mainly depends on the image contrast generated the by the individual fluorophore’s emission. Thus, measures to enhance the signal intensity or reduce the signal background allow to increase the image resolution achieved by dSTORM. In my thesis, this is achieved by simply adding a reflective metal-dielectric nano-coating to the microscopy coverslip that serves as a tunable nano-mirror. I have demonstrated that such metal-dielectric coatings provide higher photon yield at lower background and thus substantially improve SMLM performance by a significantly increased localization precision, and thus ultimately higher image resolution. The strength of this approach is that ─ except for the coated cover glass ─ no specialized setup is required. The biocompatible metal-dielectric nano-coatings are fabricated directly on microscopy coverslips and have a simple three-ply design permitting straightforward implementation into a conventional fluorescence microscope. The introduced improved lateral resolution with such mirror-enhanced STORM (meSTORM) not only allows to exceed Widefield and Total Internal Reflection Fluorescence (TIRF) dSTORM performance, but also offers the possibility to measure in a simplified setup as it does not require a special TIRF objective lens. The resolution improvement achieved with meSTORM is both spectrally and spatially tunable and thus allows for dual-color approaches on the one hand, and selectively highlighting region above the cover glass on the other hand, as demonstrated here. Beyond lateral resolution enhancement, the clear-cut profile of the highlighted region provides additional access to the axial dimension. As shown in my thesis, this allows for example to assess the three-dimensional architecture of the intracellular microtubule network by translating the local localization uncertainty to a relative axial position. Even beyond meSTORM, a wide range of membrane or surface imaging applications may benefit from the selective highlighting and fluorescence enhancing provided by the metal-dielectric nano-coatings. This includes for example, among others, live-cell Fluorescence Correlation Spectroscopy and Fluorescence Resonance Energy Transfer studies as recently demonstrated.

The advent of single molecule localization microscopy (SMLM) has transformed our capacity to investigate biological structures at the nanoscale. While the research focus has long been on improving localization precision, systematic errors caused by optical aberrations are often overlooked. In the case of 3D SMLM, such errors have the potential to significantly impair the quality of the resulting images. In this paper, we present an imaging and data processing approach that jointly estimates both, molecule positions and optical aberrations in SMLM. Therefore, the method minimizes systematic errors in SMLM reconstructions without the necessity of additional experimental calibration steps, such as the recording of fluorescent bead z-stacks. We investigate the reliability of this approach, especially in situations where the joint retrieval can be expected to be ill-posed, i.e., whenever the sample is "flat" and provides little diversity among the captured single molecule images. To enhance the reliability of the inverse problem solution, we suggest utilizing small SMLM data sets acquired at one or more slightly defocused "auxiliary" planes. We investigate the effectiveness of our approach through numerical simulations and imaging experiments of a calibration probe and nuclear pore complexes. Our method is simple and integrates seamlessly into existing SMLM setups without necessitating modifications or added complexity to the system.

Super-resolution radial fluctuation (SRRF) microscopy is a novel computational imaging technique that bypasses the optical diffraction limit (lateral resolutions of 200-300nm), achieving lateral resolutions of approximately 50-100nm while being compatible with live-cell imaging. Unlike traditional super-resolution methods such as stimulated emission depletion (STED) and single molecule localization microscopy (SMLM), SRRF minimizes phototoxicity and hardware complexity by analyzing fluorescence intensity fluctuations in standard wide-field microscopy data. This is achieved by calculating local gradient convergence ("radiality") across time-series images, enabling the reconstruction of sub-diffraction structures without specialized fluorophores or high-intensity illumination. Implemented through the open-source NanoJ-SRRF platform, SRRF optimizes parameters like ring radius and radiality magnification to enhance resolution, suppress noise, and maintain computational efficiency. Its advantages include low phototoxicity, compatibility with conventional dyes, and integration with various imaging modalities, allowing dynamic visualization of subcellular processes (e.g., mitochondrial fission, microtubule dynamics). Despite its limitations in axial resolution and potential artifacts in high-density structures, recent advancements like enhanced SRRF (eSRRF) and variance reweighted radial fluctuations and enhanced SRRF (VeSRRF) address these challenges, facilitating real-time, multicolor imaging. Applications range from ultrastructural studies to clinical pathology, with future developments in AI processing and multimodal integration promising further enhancements in imaging capabilities. SRRF stands to significantly impact the understanding of dynamic subcellular processes and biomedical research.

Single molecule localization microscopy (SMLM) is a powerful tool for imaging biological targets at the nanoscale. In this report, we present SMLM imaging of telomeres and centromeres using fluorescence in situ hybridization (FISH). The FISH probes were fabricated by decorating CdSSe/ZnS quantum dots (QDs) with telomere or centromere complementary DNA strands. SMLM imaging experiments using commercially available peptide nucleic acid (PNA) probes labeled with organic fluorophores were also conducted to demonstrate the advantages of using QDs FISH probes. Compared with the PNA probes, the QDs probes have the following merits. First, the fluorescence blinking of QDs can be realized in aqueous solution or PBS buffer without thiol, which is a key buffer component for organic fluorophores’ blinking. Second, fluorescence blinking of the QDs probe needs only one excitation light (i.e. 405 nm). While fluorescence blinking of the organic fluorophores usually requires two illumination lights, that is, the activation light (i.e. 405 nm) and the imaging light. Third, the high quantum yield, multiple switching times and a good optical stability make the QDs more suitable for long-term imaging. The localization precision achieved in telomeres and centromeres imaging experiments is about 30 nm, which is far beyond the diffraction limit. SMLM has enabled new insights into telomeres or centromeres on the molecular level, and it is even possible to determine the length of telomere and become a potential technique for telomere-related investigation.

Advanced image-free analysis of the nano-organization of chromatin and other biomolecules by Single Molecule Localization Microscopy (SMLM)

Single-molecule localization microscopy (SMLM) is a powerful technique to achieve super-resolution imaging beyond the diffraction limit. Although various types of blinking fluorophores are currently considered for SMLM, intrinsic blinking fluorophores remain rare at the single-molecule level. Here, we report the synthesis of nanographene-based intrinsic burst-blinking fluorophores for highly versatile SMLM. We image amyloid fibrils in air and in various pH solutions without any additive and lysosome dynamics in live mammalian cells under physiological conditions. In addition, the single-molecule labeling of nascent proteins in primary sensory neurons was achieved with azide-functionalized nanographenes via click chemistry. SMLM imaging reveals higher local translation at axonal branching with unprecedented detail, while the size of translation foci remained similar throughout the entire network. These various results demonstrate the potential of nanographene-based fluorophores to drastically expand the applicability of super-resolution imaging.

Single-molecule localization microscopy (SMLM) allows imaging beyond the diffraction limit. Detection of molecules is a crucial initial step in SMLM. False positive detections, which are not quantitatively controlled in current methods, are a source of artifacts that affect the entire SMLM analysis pipeline. Furthermore, current methods lack standardization, which hinders reproducibility. Here, we present an optimized molecule detection method which combines probabilistic thresholding with theoretically optimal filtering. The probabilistic thresholding enables control over false positive detections while optimal filtering minimizes false negatives. A theoretically optimal Poisson matched filter is used as a performance benchmark to evaluate existing filtering methods. Overall, our approach allows the detection of molecules in a robust, single-parameter and user-unbiased manner. This will minimize artifacts and enable data reproducibility in SMLM.

Caveolae are plasma membrane invaginations whose formation requires caveolin-1 (Cav1), the adaptor protein polymerase I, and the transcript release factor (PTRF or CAVIN1). Caveolae have an important role in cell functioning, signaling, and disease. In the absence of CAVIN1/PTRF, Cav1 forms non-caveolar membrane domains called scaffolds. In this work, we train machine learning models to automatically distinguish between caveolae and scaffolds from single molecule localization microscopy (SMLM) data. We apply machine learning algorithms to discriminate biological structures from SMLM data. Our work is the first that is leveraging machine learning approaches (including deep learning models) to automatically identifying biological structures from SMLM data. In particular, we develop and compare three binary classification methods to identify whether or not a given 3D cluster of Cav1 proteins is a caveolae. The first uses a random forest classifier applied to 28 hand-crafted/designed features, the second uses a convolutional neural net (CNN) applied to a projection of the point clouds onto three planes, and the third uses a PointNet model, a recent development that can directly take point clouds as its input. We validate our methods on a dataset of super-resolution microscopy images of PC3 prostate cancer cells labeled for Cav1. Specifically, we have images from two cell populations: 10 PC3 and 10 CAVIN1/PTRF-transfected PC3 cells (PC3-PTRF cells) that form caveolae. We obtained a balanced set of 1714 different cellular structures. Our results show that both the random forest on hand-designed features and the deep learning approach achieve high accuracy in distinguishing the intrinsic features of the caveolae and non-caveolae biological structures. More specifically, both random forest and deep CNN classifiers achieve classification accuracy reaching 94% on our test set, while the PointNet model only reached 83% accuracy. We also discuss the pros and cons of the different approaches.

As one of the super-resolved optical imaging techniques, single molecule localization microscopy (SMLM) received considerable attention due to its impressive spatial resolution. Compared with other fluorescence imaging techniques, SMLM has one particular request for the fluorophores, that is, continuous ‘on’ and ‘off’ behaviors of their signals (referred to as ‘blinking’). Hence, we present here a kind of super blinking and biocompatible nanoprobes (denoted as SBNs) for SMLM. The SBNs have two main advantages, first, they possess an outstanding fluorescence blinking. Second, they are biocompatible since they are based on bovine serum albumin (BSA). The SBNs are fabricated by doping organic dyes into BSA nanoparticles. We fabricated two kinds of SBNs, one was doped with Alexa Fluor 647 (A647) and the other was doped with Alexa Fluor 594 (A594). Especially for A594 doped SBNs, the improved blinking of A594 doped SBNs induced a better localization precision as compared with A594 alone. Moreover, SMLM imaging of breast cancer cells and exosomes using the SBNs was successfully realized with high spatial resolutions. The work demonstrated here provides a new strategy to prepare novel kinds of super blinking fluorescent agents for SMLM, which broadens the selection of suitable fluorophores for SMLM.

To better understand the interactions between biological molecules, a high optical resolution in all three dimensions is crucial. The intrinsically lower axial resolution of microscopes however, is a limiting factor in fluorescence imaging, correspondingly in fluorescence based single molecule localization microscopy (SMLM). Here, we present a method to improve the axial localization precision in SMLM by combining point-spread-function engineering with total internal reflection fluorescence (TIRF) fields with decay lengths that vary within the on-time of a fluorophore. Such time-varying illumination field intensity allows one to extract additional axial location information from the emitted photons. With this time varying illumination approach, we show that axial localization is improved two-fold over TIRF-based SMLM using astigmatic PSFs. We calculate theoretical resolution gains for various imaging conditions via the Cramér Rao Lower Bound (CRLB), a commonly used metric to compute the best attainable localization precision in SMLM.

Single molecule localization microscopy is a recently developed superresolution imaging technique to visualize structural properties of single cells. The basic principle consists in chemically attaching fluorescent dyes to the molecules, which after excitation with a strong laser may emit light. To achieve superresolution, signals of individual fluorophores are separated in time. In this paper we follow the physical and chemical literature and derive mathematical models describing the propagation of light emitted from dyes in single molecule localization microscopy experiments via Maxwell’s equations. This forms the basis of formulating inverse problems related to single molecule localization microscopy. We also show that the current status of reconstruction methods is a simplification of more general inverse problems for Maxwell’s equations as discussed here.

Chapter Thirteen - Localization Microscopy

Cancer is a major health threat worldwide. Options for targeted cancer therapy, however, are often limited, in a large part due to our incomplete understanding of how key processes including oncogenesis and drug response are mediated at the molecular level. New imaging techniques for visualizing biomolecules and their interactions at the nanometer and single molecule scales, collectively named fluorescence nanoscopy, hold the promise to transform biomedical research by providing direct mechanistic insight into cellular processes. We discuss the principles of quantitative single-molecule localization microscopy (SMLM), a subset of fluorescence nanoscopy, and their applications to cancer biomedicine. In particular, we will examine oncogenesis and drug resistance mediated by mutant Ras, which is associated with ~1/3 of all human cancers but has remained an intractable drug target. At ~20 nm spatial and single-molecule stoichiometric resolutions, SMLM clearly showed that mutant Ras must form dimers to activate its effector pathways and drive oncogenesis. SMLM further showed that the Raf kinase, one of the most important effectors of Ras, also forms dimers upon activation by Ras. Moreover, treatment of cells expressing wild type Raf with Raf inhibitors induces Raf dimer formation in a manner dependent on Ras dimerization. Together, these data suggest that Ras dimers mediate oncogenesis and drug resistance in tumors with hyperactive Ras and can potentially be targeted for cancer therapy. We also discuss recent advances in SMLM that enable simultaneous imaging of multiple biomolecules and their interactions at the nanoscale. Our work demonstrates the power of quantitative SMLM in cancer biomedicine.

A well-reasoned model of data movie in single molecule localization microscopy (SMLM) is desired. A model of data movie can be decomposed into a model of emitter activation process and a model of data frame. In this paper, we focus on Markov chain modeling and analyzing of emitter activation process for both cycled and continuous illuminations. First, a two-phase Markov chain is proposed to model the activation process for a pair of conjugated activator and emitter with cycled illumination. By converting the frame-based Markov chain into several cycle-based Markov chains, the stationary state distribution in the photoactivatable period is derived. Further obtained are several formulas that capture the characterization of the two-phase Markov chain. Second, the Markov chain and analytical result are extended to the continuous illumination where an emitter is excited continuously in all frames. Finally, incorporating the model of emitter activation process with our previous model of data frame, the model of data movie for both cycled and continuous illuminations in 3D and 2D imaging are simulated by custom codes. It is shown that the model can synthesize data movies well and the analytical formulas predict the simulation results accurately. The models provide a means to be broadly utilized in generating well-reasoned data movies for training of neural networks and evaluation of localization algorithms.