Fourier ptychography microscopy for digital pathology.

Fourier ptychographic microscopy (FPM) emerged as a prominent imaging technique in 2013, attracting significant interest due to its remarkable features such as precise phase retrieval, expansive field of view (FOV), and superior resolution. Over the past decade, FPM has become an essential tool in microscopy, with applications in metrology, scientific research, biomedicine, and inspection. This achievement arises from its ability to effectively address the persistent challenge of achieving a trade-off between FOV and resolution in imaging systems. It has a wide range of applications, including label-free imaging, drug screening, and digital pathology. In this comprehensive review, we present a concise overview of the fundamental principles of FPM and compare it with similar imaging techniques. In addition, we present a study on achieving colorization of restored photographs and enhancing the speed of FPM. Subsequently, we showcase several FPM applications utilizing the previously described technologies, with a specific focus on digital pathology, drug screening, and three-dimensional imaging. We thoroughly examine the benefits and challenges associated with integrating deep learning and FPM. To summarize, we express our own viewpoints on the technological progress of FPM and explore prospective avenues for its future developments.

Fourier ptychographic microscopy (FPM) is a computational approach geared towards creating high-resolution and large field-of-view images without mechanical scanning. Acquiring color images of histology slides often requires sequential acquisitions with red, green, and blue illuminations. The color reconstructions often suffer from coherent artifacts that are not presented in regular incoherent microscopy images. As a result, it remains a challenge to employ FPM for digital pathology applications, where resolution and color accuracy are of critical importance. Here we report a deep learning approach for performing unsupervised image-to-image translation of FPM reconstructions. A cycle-consistent adversarial network with multiscale structure similarity loss is trained to perform virtual brightfield and fluorescence staining of the recovered FPM images. In the training stage, we feed the network with two sets of unpaired images: (1) monochromatic FPM recovery and (2) color or fluorescence images captured using a regular microscope. In the inference stage, the network takes the FPM input and outputs a virtually stained image with reduced coherent artifacts and improved image quality. We test the approach on various samples with different staining protocols. High-quality color and fluorescence reconstructions validate its effectiveness.

Adaptive denoising method for Fourier ptychographic microscopy

White blood cells (WBC) play a significant role in human immune system, so WBC detection is a meaningful work. In this paper, we propose a novel framework which combines Fourier ptychographic microscopy (FPM) and SO-you only look once (YOLO) for WCB detection. FPM is a recently developed microscope technology which can achieve high-resolution, wide-field, and quantitative phase imaging at the same time without mechanical moving and phase measurement. With the FPM, we can get high resolution, wide field-of-view blood cell images at one time. After obtaining high-resolution images, we propose SO-YOLO as our detection method. As a convolutional neural network, the SO-YOLO outperforms the state-of-the-art detection methods both in accuracy and speed. The proposed framework in this paper can be used in clinical diagnose, because the experiment setup is flexible and the detection is fast. The data set in this paper is made with FPM by ourselves. Our data set contains 1000 color high resolution images and all the WBC has been detected and counted by a human expert. The detection results by the experts are regarded as the ground truth. The experiment results show that the SO-YOLO performs well in detecting small objects compared with other methods.

Color Fourier ptychographic microscopy (FPM), particularly for high-throughput applications like digital pathology, is fundamentally limited by the trade-off between acquisition speed (sampling density) and reconstruction fidelity. To address this, we propose a novel, to the best of our knowledge, physics-guided deep learning framework for high-fidelity color FPM reconstruction under low-frequency spectrum acquisition. The core of our approach is the deep integration of the FPM physical model into a generative diffusion model, which effectively guides the model to extrapolate high-frequency spectral information beyond the captured synthetic aperture. This hybrid physical-generative approach enables robust image reconstruction from significantly reduced acquisition data, achieving high-quality results using illumination restricted to the bright-field region of the objective lens. Both simulations and experiments demonstrate that our method significantly outperforms existing traditional and deep learning-based approaches in recovering fine structural details while suppressing artifacts, without compromising computational efficiency. This physics-guided deep learning paradigm provides a novel strategy for high-speed, high-fidelity computational microscopy with significant potential for clinical translation.

Phase retrieval is an important tool to unveil wavefront of light, especially in high performance microscopy such as Fourier ptychographic microscopy (FPM). In general phase-retrieval methods, the resolution and the number of measurements are in a trade-off relationship. Inspired by FPM, we devise what we believe is a novel microscopic phase-retrieval method, termed single-shot FPM (SSFPM). In our approach, the imaging performance exceeds the trade-off relationship in that it enables phase retrieval for high resolution with a single measurement. By placing the lens array at the Fourier plane of the objective lens, multiple intensity profiles required for the FPM algorithm are collected in a single shot. To achieve enough redundancy of data for satisfying convergence condition of FPM, the specimen is simultaneously illuminated by multiple light-emitting diodes. SSFPM reconstructs quantitative phase profile and enhances the resolution sacrificed by applying lens-array imaging. We demonstrate the performance of SSFPM with numerical simulation and experiments. The prototype achieves lateral resolution of 3.10 μm over a field of view of 0.34 mm2. Without an interferometer or scanning devices, SSFPM can reconstruct high resolution of a complex profile with a single shot.

The integration of neural network and physical reconstruction model for Fourier ptychographic microscopy

Image stacks provide invaluable 3D information in various biological and pathological imaging applications. Fourier ptychographic microscopy (FPM) enables reconstructing high-resolution, wide field-of-view image stacks without z-stack scanning, thus significantly accelerating image acquisition. However, existing FPM methods take tens of minutes to reconstruct and gigabytes of memory to store a high-resolution volumetric scene, impeding fast gigapixel-scale remote digital pathology. While deep learning approaches have been explored to address this challenge, existing methods poorly generalize to novel datasets and can produce unreliable hallucinations. This work presents FPM-INR, a compact and efficient framework that integrates physics-based optical models with implicit neural representations (INRs) to represent and reconstruct FPM image stacks. FPM-INR is agnostic to system design or sample types and does not require external training data. In our experiments, FPM-INR substantially outperforms traditional FPM algorithms with up to a 25-fold increase in speed and an 80-fold reduction in memory usage for continuous image stack representations.

Fourier ptychographic microscopy (FPM) is an attractive method to extend the resolution beyond the conventional limit defined by a microscope optics, sharing properties with ptychographic, synthetic aperture imaging and phase retrieval. The algorithm uses a sequence of low-resolution (LR) images acquired under angularly varying illumination to reconstruct a high-resolution (HR) image. However, traditional FPM may trap to the sub-optimal solution, since brute-force replacement in the Fourier domain is applied. To address this problem, we propose here a weighted replacement for Fourier ptychographic microscopy (WFPM). We employ the weighted average of spectrums corresponding to different illumination angle to replace the overlapped regions in the Fourier domain. A series of experimental results demonstrate that the reconstructed image using the proposed WFPM shows a better quality and a faster convergence compared with the results obtained by FPM.

As a typical computational method, Fourier ptychographic microscopy (FPM) can realize high spatial resolution and quantitative phase imaging while preserving the large field of view with a low numerical aperture (NA) objective. A programmable light-emitting diode (LED) array is used as a typical illuminator in an FPM system, and the illumination parameters of each LED element are crucial to the success of the FPM reconstruction algorithm. Compared with LED arrays arranged in rectangular arrays, LED arrays with special structures such as domes or rings can effectively improve FPM imaging results and imaging efficiency. As a trade-off, their calibration difficulty is greatly increased due to the lack of geometric constraints of rectangular arrays. In this paper, we propose an effective hybrid full-pose parameter calibration method for freeform LED array illuminators, combining stereoscopic 3D imaging techniques and the geometric constraints of the microscopic platform. First, a stereovision system is used to obtain the accurate 3D position of each LED element of the freeform illuminator and to construct a rigid 3D coordinate LED array system. Then, calibration between the coordinate system of the LED array and that of the optical imaging component is realized according to the geometric features of the brightfield-to-darkfield edges. Finally, we verify the feasibility and effectiveness of the proposed method through full-pose parameter calibration of LED arrays with different arrangement rules.

Fourier ptychography microscopy(FPM) is a recently developed computational imaging approach for microscopic super-resolution imaging. By turning on each light-emitting-diode (LED) located on different position on the LED array sequentially and acquiring the corresponding images that contain different spatial frequency components, high spatial resolution and quantitative phase imaging can be achieved in the case of large field-of-view. Nevertheless, FPM has high requirements for the system construction and data acquisition processes, such as precise LEDs position, accurate focusing and appropriate exposure time, which brings many limitations to its practical applications. In this paper, inspired by artificial neural network, we propose a Fourier ptychography multi-parameter neural network (FPMN) with composite physical prior optimization. A hybrid parameter determination strategy combining physical imaging model and data-driven network training is proposed to recover the multi layers of the network corresponding to different physical parameters, including sample complex function, system pupil function, defocus distance, LED array position deviation and illumination intensity fluctuation, etc. Among these parameters, LED array position deviation is recovered based on the features of brightfield to darkfield transition low-resolution images while the others are recovered in the process of training of the neural network. The feasibility and effectiveness of FPMN are verified through simulations and actual experiments. Therefore FPMN can evidently reduce the requirement for practical applications of FPM.

Fourier ptychographic microscopy (FPM) is a computational imaging technique developed in recent years. The validity of the final reconstruction results is closely related to many systematic errors, among which the pupil aberration is a key one. Although the traditional FPM can recover the pupil function in the iterative process, it increases the dependence on data redundancy invisibly, and cannot satisfy the high-speed and low redundancy FPM reconstruction with few images. In this paper, a pupil function recovering method based on annular illumination is proposed, which only uses a small number of LEDs matching the NAobj for illumination. In addition, the Zernike polynomial is used to constrain the freedom degree of aberration, to ensure that the object and pupil can be recovered correctly with fewer data. The simulation results show that the proposed method can recover and correct the pupil function within 6 annular bright-field images, which greatly reduces the requirement of data redundancy and provides the possibility for high-speed aberration recovery based on FPM.

Fourier ptychographic microscopy (FPM) is a computational imaging technique that achieves high-resolution complex amplitude reconstruction across a large field of view. However, conventional FPM is fundamentally limited to regions near the optical axis due to violations of the shift-invariance assumption in off-axis areas, resulting in challenging edge-of-field-of-view reconstructions. We propose neural pupil engineering FPM, termed NePE-FPM, a physical model that dynamically shifts the pupil function position during reconstruction instead of fixing the pupil function in the center of the pupil plane. NePE-FPM engineers the pupil function using an implicit neural representation with multi-resolution hash encoding, enabling continuous, smooth shifting of the pupil function without introducing additional physical parameters. By optimizing a feature-domain loss function, NePE-FPM adaptively filters Fourier-space information from low-resolution measurements, achieving accurate off-axis reconstruction without modeling off-axis propagation. Experimental results demonstrate isotropic resolution of 1149 lp/mm across an 11mm2 FOV using a 4×/0.1NA objective. The NePE-FPM bridges the gap between theoretical FPM capabilities and practical whole-slide imaging demands.

I will report on a new computational method for microscopy, Fourier Ptychographic Microscopy. This method performs phase retrieval through illumination angle diversity and allows us to transform a standard microscope with a 0.08 NA objective into a high-performance gigapixel microscope with 0.8 micron resolution, field-of-view of 120 mm2 and a long depth-of-field of 0.3 mm. This method transform the general challenge of standard microscopy from one coupled to the physical limitations of the system's optics to one that is computationally solvable. The method can significantly simplify digital pathology procedures.

Fourier ptychographic microscopy (FPM) is a recently proposed computational imaging method that achieves both high resolution (HR) and wide field of view. In the conventional FPM model, the sample is assumed to be a 2D thin layer, and a series of low-resolution images at different illumination angles is used for HR image reconstruction. However, in practice, the sample complex amplitude distribution is usually mixed with some unknown background interferences. These background interferences may result from inhomogeneous media distribution or other defocusing layers, etc. The background interference will significantly degrade FPM reconstruction results, but so far it is not considered in the conventional FPM algorithm. Here, we propose a method that adaptively separates background interferences for each illumination angle. Experimental results show that the proposed method has a faster convergence speed and better reconstruction accuracy than the conventional FPM model.