Single pixel reconstruction imaging: Taking confocal imaging to the extreme

This doctoral thesis addresses two novel approaches in fluorescence microscopy. The first part introduces a new method that can be used to enhance the effect of photoselection. This method, termed ExPAN (excitation polarization angle narrowing), exploits stimulated emission to narrow the angular distribution of excited fluorophores. ExPAN could be of significant interest for applications like fluorescence anisotropy measurements, which are commonly applied in life science, e.g. to study receptor-ligand interactions or investigate protein dynamics. In the context of this work, ExPAN was applied in conjunction with a new fluorescence microscopy approach that utilizes molecular orientation as a contrast mechanism. Using excitation light with continuously rotating polarization, it is possible to obtain information about the orientation of dye molecules. When excited in this manner, molecules with a preferred orientation exhibit a periodically modulated fluorescence signal as a result of photoselection. This work showed that the additional modulation information can be utilized to distinguish molecules with different orientations even when their signals overlap spatially. The approach was explored by experiments with single molecules on a surface using a modified widefield microscope and additionally by simulation studies. The results indicated that it is possible to distinguish signals from dye molecules with a distance down to 80 nm. It was demonstrated that ExPAN enhances the effect of photoselection and can be used to distinguish molecules even when their orientations are very similar. Furthermore, a modulated fluorescence signal was also observed in fixed biological samples and with surface labeled microparticles in aqueous solution. In the second part of this thesis a method is presented for improving the resolution of confocal laser scanning microscopes. This method, termed multi-detector scanning (MDS), is based on the concept of image scanning microscopy (ISM). Using ISM it is theoretically possible to double the resolution of a fluorescence microscope. Since ISM requires an array detector, most of the previously developed implementations use CCD or CMOS cameras for detection. Instead of a camera, several single-photon detectors that are combined to an array detector by a fiber optic bundle, are used in this work.This configuration allowed it for the first time to use the method in conjunction with Fluorescence-lifetime imaging microscopy (FLIM). FLIM has proven to be an important microscopy technique in life science and has been applied in a wide variety of studies including protein-protein interaction and cell metabolism research. Therefore, the improvement in spatial resolution of FLIM through ISM is of potential interest for numerous biological applications. Within the scope of this work, a multi-detector scanning microscope was constructed and characterized. Resolution improvements of 168 nm with MDS and 146 nm with MDS and deconvolution were achieved in fixed biological samples. Furthermore, the measurement of fluorescence lifetimes was demonstrated with the experimental setup. Thus, the application of MDS has the potential to increase the spatial resolution of FLIM and could serve as a tool for measuring biological systems in more detail.

Image scanning microscopy (ISM) is a promising tool for bioimaging owing to its integration of signal to noise ratio (SNR) and super resolution superior to that obtained in confocal scanning microscopy. In this paper, we introduce the annular radially polarized beam to the ISM, which yields an axially extended excitation focus and enhanced resolution, providing a new possibility to obtain the whole information of thick specimen with a single scan. We present the basic principle and a rigorous theoretical model for ISM with annular radially polarized beam (ISM-aRP). Results show that the resolution of ISM-aRP can be enhanced by 4% compared with that in conventional ISM, and the axial extent of the focus is longer than 6λ. The projected view of the simulated fluorescent beads suspension specimen demonstrates the validity of ISM-aRP to obtain the whole information of volume sample. Moreover, this simple method can be easily integrated into the commercial laser scanning microscopy systems.

Super-resolution imaging is a powerful tool in modern biological research, allowing for the optical observation of subcellular structures with great detail. In this paper, we present a deep learning approach for image fusion of intensity and super-resolution optical fluctuation imaging (SOFI) microscopy images. We construct a network that can successfully combine the advantages of these two imaging methods, producing a fused image with a resolution comparable to that of SOFI and an SNR comparable to that of the intensity image. We also demonstrate the effectiveness of our approach experimentally, specifically on cell samples where microtubules were stained with ATTO647N and imaged using a confocal microscope with a single photon fiber bundle camera, allowing for the simultaneous acquisition of an image scanning microscopy (ISM) image and a SOFISM (ISM and SOFI) image. Our network is designed as a self-supervised network and shows the ability to train on a single pair of images and to generalize to other image pairs without the need for additional training. Our approach offers a flexible and efficient way to combine the strengths of correlation based imaging techniques along with traditional intensity based microscopy, and can be readily applied to other fluctuation based imaging modalities.

High-speed 3D optical microscopy is an indispensable requirement in studying rapid processes like signaling in neuronal networks, flagellar motion, or complex motion of living and highly dynamic subcellular components. However, the 3D imaging capability often compromises either the acquisition speed or the complexity of the imaging system. Multi-plane imaging offers a parallelized acquisition of several focal planes at the same time, which tremendously enhances the temporal resolution. Nonetheless, most of the current multi-plane approaches are complex and require several optical elements to create multiple beams with different optical path lengths, limiting their broad application. To address these problems, in this thesis, employing a multi-plane prism, we introduce three novel techniques that each improve the concept of rapid 3D optical microscopy in a different direction: First, we accomplish a multi-plane detection for a phase-contrast microscope resulting in a 3D, label-free, and non-complex imaging system with a ~ 4ms of temporal resolution. We applied this system in real-time studying isolated bare axonemes of Chlamydomonas beating in the vicinity of a surface. This enabled us to observe their non-zero torsional motion where the torsion sign slowly changes from negative at the basal end toward positive at the distal end of flagella. Second, straightforwardly, we combine an experimental spectral unmixing setup with the multi-plane detection and obtain an instant 3- color 3D microscope with a minimal axial color aberration of 140 nm in a ~ 2.5 μm axial range. The performance of such a microscope is verified in volumetric imaging of three different subcellular components in fixed COS-7 cells. Third, a single-color multi-plane fluorescence microscope has been employed as a rapid 3D particle tracking velocimetry method in microscale to study single particles tracing the so-called Marangoni flow. Using that, besides the velocity fields of single particles, we achieve a localization precision of ~ 60 nm in the lateral and ~ 300 nm in the axial direction over a 120 120 13 m3 field of view. Altogether, using these three techniques, we extend the temporal resolution, throughput, and applicability of 3D optical microscopy. In another project, we implemented image scanning microscopy simultaneously in two colors on a conventional wide-field microscope by applying an LED light source and a digital micro-mirror device. This permits us to realize an inexpensive, speckle noise-free, and easy-to-implement super-resolution add-on for a conventional epi-fluorescence microscope.

For almost a century, the resolution of optical microscopy was thought to be limited by Abbé's law describing the diffraction limit of light. At the turn of the millennium, aided by new technologies and fluorophores, the field of optical microscopy finally surpassed the diffraction barrier: a milestone achievement that has been recognized by the 2014 Nobel Prize in Chemistry. Many super-resolution methods rely on the unique photophysical properties of the fluorophores to improve resolution, posing significant limitations on biological imaging, such as multicoloured staining, live-cell imaging and imaging thick specimens. Structured Illumination Microscopy (SIM) is one branch of super-resolution microscopy that requires no such special properties of the applied fluorophores, making it more versatile than other techniques. Since its introduction in biological imaging, SIM has proven to be a popular tool in the biologist's arsenal for following biological interaction and probing structures of nanometre scale. SIM continues to see much advancement in design and implementation, including the development of Image Scanning Microscopy (ISM), which uses patterned excitation via either predefined arrays or raster-scanned single point-spread functions (PSF). This review aims to give a brief overview of the SIM and ISM processes and subsequent developments in the image reconstruction process. Drawing from this, and incorporating more recent achievements in light shaping (i.e. pattern scanning and super-resolution beam shaping), this study also intends to suggest potential future directions for this ever-expanding field.

Circular SAR (CSAR) has the ability of 3-D imaging due to its special curve trajectory. Single-pass CSAR can theoretically obtain the resolution of the sub-wavelength level on the distance-azimuth plane, but its resolution at the elevation direction is very low. At the same time, CSAR 3-D imaging with Back Projection(BP) has high algorithm complexity and low imaging efficiency. A coherent 3-D imaging method for multi-circular SAR based on an improved 3-D back projection algorithm is proposed. For the problem of high time complexity of the imaging algorithm, an improved 3-D BP algorithm for CSAR based on constructing geometric interpolation kernel is proposed. 3-D interpolation operations are transformed into 1-D interpolation operations and distance vector searching operations. The final imaging result is obtained by coherently accumulating the improved 3-D BP results of multi-circular SAR. The proposed method solves effectively the problem of low elevation resolution of single-pass CSAR, improves 3-D imaging details, and reduces greatly the time of CSAR 3-D imaging simultaneously. The simulated 3-D imaging results of the conical target and GOTCHA data set from the US Air Force Laboratory verify the effectiveness of the proposed method.

The application of confoca]. imaging systems to image the cornea and the ocular lens of the living eye is a major improvement in ophthalmic imaging. This new application of confocal imaging to the living eye has potential for both basic research and clinical diagnostics. The standard optical imaging systems have poor resolution and low contrast when applied to ocular tissue. The use of confocal systems to image the nearly transparent cornea and ocular lens has several advantages as an imaging system. The two important physical improvements are: (1) almost total rejection of out- offocal plane reflected light, and (2) increased lateral resolution. The problems of ocular imaging with standard slit lamps and specular microscopes include: low contrast images due to contributions to the image from out of focal plane reflections, and low resolution. Confocal imaging systems can image living eyes. The images show sharp, high resolution, high contrast features of submicron structures. Examples of ocular confocal images include the following: submicron optical sections through the various cells in the cornea, nerve fibers, cellular processes, and images through the nucleus of living cells comprising the cornea and the ocular lens. The ocular lens has been imaged. The lens capsule, lens epithelial cells, lens fibers and nuclei are readily observed. The use of realtime confocal imaging systems in ocular medical imaging has resulted in improved quality (resolution and contrast) of cellular images of living eyes. Further developments require the availability of long free working distance objectives with a high numerical aperture. This paper demonstrates the advantages of imaging ocular tissue with a confocal imaging system. Confocal imaging of ophthalmic structures may become a standard clinical tool with the new developments in instrumentation.

The small number of pixels in the current linear mode avalanche photodiode (LM-APD) array limits its three-dimensional (3D) imaging resolution. We use an optical phased array-based beam array subdivided pixel method to improve the 3D imaging resolution, using an optical phased array to generate a beam array with the same number of pixels as the LM-APD array and matching positions and controlling each sub-beam in the beam array to scan in the field of view of the corresponding pixel. The sub-beam divergence angle in the beam array is smaller than the instantaneous field of view angle of a single pixel in the LM-APD array. The sub-beam scanning in a single pixel’s field of view realizes the multiple acquisition of the target 3D information by the LM-APD array, thus improving the resolution of the LM-APD array. The distribution of the beam array in the far field is simulated, and the main performance parameters of 3D imaging are analyzed. Finally, a liquid crystal phase modulator is used as an optical phased array device to conduct experiments on a target 20 m away, and the results prove that our method can improve the resolution from 4 × 4 to 8 × 8, which can be improved at least four times.

Fast, nondestructive three-dimensional (3D) imaging of live suspension cells remains challenging without substrate treatment or fixation, precluding scalable single-cell morphometry with minimal alterations. While optical sectioning techniques achieve 3D live cell imaging, lateral versus depth resolution differences further complicate analysis. We present a scalable microfluidic method capable of 3D fluorescent isotropic imaging of live, nonadherent cells suspended inside picoliter droplets with high-speed single-cell volumetric readout (800 to 1,200 slices in 5 to 8 s) and near-diffraction limit resolution (~216 nm). The platform features a droplet trap array that leverages flow-induced droplet interfacial shear to generate intradroplet microvortices, which rotate single cells on their axis to enable optical projection tomography (OPT)-based imaging. This allows gentle (~1 mPa shear stress) observation of cells encapsulated inside nontoxic isotonic buffer droplets, facilitating scalable OPT acquisition by simultaneous spinning of hundreds of cells. We demonstrate 3D imaging of live myeloid and lymphoid cells in suspension, including K562 cells, as well as naive and activated T cells-small cells prone to movement in their suspended phenotype. Our fully suspended, orientation-independent cell morphometry, driven by isotropic imaging and spherical harmonic analysis, enabled the study of primary T cells across various immunological activation states. This approach unveiled six distinct nuclear content distributions, contrasting with conventional 2D images that typically portray spheroid and bean-like nuclear shapes associated with lymphocytes. Our arrayed-droplet OPT technology is capable of isotropic, single live-cell 3D imaging, with the potential to perform large-scale morphometry of immune cell effector function states while providing compatibility with microfluidic droplet operations.

We introduce a compressive sensing based approach for single pixel hyperspectral chemical imaging in a broad spectral range in the near-infrared. Fully integrated MEMS based Fabry-Pérot tunable filter spectrometers and a digital micro-mirror device were employed to achieve spectral and spatial resolution, respectively. The available spectral range from 1500 to 2200 nm covers molecular overtone vibrations enabling chemical identification. Hyperspectral images of different adhesives deposited on a textile were recorded revealing their chemical composition. Furthermore, spectrally resolved near-infrared images with compression rates up to 90% are presented. The approach of single pixel imaging illustrates a promising technology for the infrared spectral range superior to conventionally used costly focal plane arrays.

Purpose 3 D imaging of the lung is not a trivial undertaking as during preparation the lung may collapse. Also serial sections and scans followed by 3 D reconstruction may lead to artifacts. The present study aims to figure out the best way to perform 3 D imaging in lung research. Materials and Methods We applied an optical tissue clearing (OTC) method, which uses ethyl cinnamate (ECi) as a fast, nontoxic and cheap clearing solvent, for 3 D imaging of retrograde perfused lungs by laser confocal fluorescence microscopy and light sheet fluorescence microscopy. We also introduced expansion microscopy (ExM), a cutting-edge technique, in 3 D imaging of lungs. We examined and compared the usefulness of these techniques for 3 D lung imaging. The ExM protocol was further extended to paraffin-embedded lung metastases blocks. Results The MHI148-PEI labeled lung vasculature was visualized by retrograde perfusion combined with trachea ligation and ECi based OTC. As compared with trans-cardiac perfusion, the retrograde perfusion results in a better maintenance of lung morphology. 3 D structure of alveoli, vascular branches and cilia in lung were revealed by immunofluorescence staining after ExM. 3 D distribution of microvasculature and neutrophil cells in 10 years old paraffin-embedded lung metastases were analyzed by ExM. Conclusions The retrograde perfusion combined with trachea ligation technique could be applied in the lung research in mice. 3 D structure of lung vasculature can be visualized by MHI148-PEI perfusion and ECi based OTC in an efficient way. ExM and immunofluorescence staining protocol is highly recommended to perform 3 D imaging of fresh fixed lung as well as paraffin-embedded lung blocks.

Investigations on nanomedicine involve conventional two dimensional (2D) imaging techniques for observing the nanoparticle internalization at a single time point where various phases of internalization can be overlooked. In contrast, three dimensional (3D) imaging of fluorescent nanoparticles with anticancer potential can be used for obtaining the time course of cellular retention of particles, and cells can be followed for days. This article demonstrates the application of laser scanning confocal microscopy to quantify poly‐l‐lysine coated fluorescent ZnO nanoparticle retention and reactive oxygen species (ROS) generation using volumetric imaging. Synthesis of these particles allows monitoring of ROS formation, internalization, and cytotoxicity using the same imaging platform that offers an advantage over measurement using various instruments. PLL‐coated ZnO particles' ability to induce a significant reduction in cell‐viability suggests its potential as a therapeutic agent. The proposed framework opens up a new avenue for investigating mechanistic aspects of ZnO adsorption and the evaluation of therapeutic efficiency.

Single-pixel imaging could be a superior solution for imaging applications where the detector array is very expensive or not even available. Sampling order, sampling ratio, noise and type of transforms affect the quality of the reconstructed image. Here, we compare the performance of single pixel imaging (SPI) with Hadamard transform (HT) and discrete cosine transform (DCT) in the presence of noise. The trade-off between adding image information and adding noise in each coefficient measurement results in an optimum number of measurements for reconstruction image quality. In addition, DCT shows higher image quality with fewer measurements than HT does. We then demonstrate our SPI with optimum sampling strategy for a large set of images and lab experiments and finally put forward a quality control technique, which is corroborated by the practical experiments. Our results suggest a practical approach for SPI to improve the speed and achieve the highest possible image quality.

In this manuscript, the novel three dimensional (3D) image reconstruction approach based on affinity propagated clustering aided computerized Inherent Seeded Region Growing and Deep learned Marching Cubes Algorithm (ISRG‐DMCA) is proposed. The major purpose of this manuscript is to divide the brain tumor based on Shapelets. Here, the information about the shape/depth that can be obtained in every two dimensional (2D) image on the image stack is handled to acquire a 3D reconstruction, which provides high accurate 3D view of tumor Region of Interest (ROI). Then, the 3D model is rendered with the help of the proposed Deep learned Marching Cubes Algorithm (Deep MCA) at 3D reconstruction technique. The performance of the proposed method is executed in MATLAB. The simulation results show that the proposed ISRG‐DMCA algorithm attains a higher detection rate 14.117%, 5.435%, higher accuracy rate 9.556%, 26.41% and lower execution time 66.667%, 75%, compared with the existing methods, like Improved Marching Cubes Algorithm (IMCA), Improved CNN‐CRF method, respectively. In the proposed ISRG‐DMCA method, the volume of the tumor has a length of 2.56 mm. Finally, the simulation outcomes demonstrate that the proposed ISRG‐DMCA method can be able to find the optimal solutions efficiently and accurately.

Multiview display is a popular method to deliver three-dimensional (3D) images by generating a perspective directional view. However, there are some limitations such as a low resolution, lack of motion parallax, and a narrow viewing angle. In this paper, we propose a method to implement a multi-view display system that provides a 3D image in high resolution. The original setup is composed of a stereoscopic 3D display panel and a head tracking camera. The directional view image of a 3D object is captured by a camera array and shown on a stereoscopic 3D display. A user interface is designed to control the hardware. An Intel RealSense sr300 camera is used to track the observer's viewing angle. The images are captured rotationally through a movable camera array in a 30-degree span. There are 71 and 3 views in the horizontal and vertical direction, respectively. The directional view information is displayed according to the observer's viewing direction as well as the head position. The observer can realize a high-resolution 3D image with smooth motion parallax. Most importantly, the proposed system interactively displays the exact view direction according to the user's viewing angle which feels more natural to the observer.