Adaptive denoising method for Fourier ptychographic microscopy

Fourier ptychographic microscopy (FPM) is a newly developed microscopic technique for large field of view, high-resolution and quantitative phase imaging by combining the techniques from ptychographic imaging, aperture synthesizing and phase retrieval. In FPM, an LED array is utilized to illuminate the specimen from different angles and the corresponding intensity images are synthesized to reconstruct a high-resolution complex field. As a flexible and low-cost approach to achieve high-resolution, wide-field and quantitative phase imaging, FPM is of enormous potential in biomedical applications such as hematology and pathology. Conventionally, the FPM reconstruction problem is solved by using a phase retrieval method, termed Alternate Projection. By iteratively updating the Fourier spectrum with low-resolution-intensity images, the result converges to a high-resolution complex field. Here we propose a new FPM reconstruction framework with deep learning methods and design a multiscale, deep residual neural network for FPM reconstruction. We employ the widely used open-source deep learning library PyTorch to train and test our model and carefully choose the hyperparameters of our model. To train and analyze our model, we build a large-scale simulation dataset with an FPM imaging model and an actual dataset captured with an FPM system. The simulation dataset and actual dataset are separated as training datasets and test datasets, respectively. Our model is trained with the simulation training dataset and fine tuned with the fine-tune dataset, which contains actual training data. Both our model and the conventional method are tested on the simulation test dataset and the actual test dataset to evaluate the performances. We also show the reconstruction result of another neural network-based method for comparison. The experiments demonstrate that our model achieves better reconstruction results and consumes much less time than conventional methods. The results also point out that our model is more robust under system aberrations such as noise and blurring (fewer intensity images) compared with conventional methods.

High-throughput quantitative phase imaging (QPI) is essential to cellular phenotypes characterization as it allows high-content cell analysis and avoids adverse effects of staining reagents on cellular viability and cell signaling. Among different approaches, Fourier ptychographic microscopy (FPM) is probably the most promising technique to realize high-throughput QPI by synthesizing a wide-field, high-resolution complex image from multiple angle-variably illuminated, low-resolution images. However, the large dataset requirement in conventional FPM significantly limits its imaging speed, resulting in low temporal throughput. Moreover, the underlying theoretical mechanism as well as optimum illumination scheme for high-accuracy phase imaging in FPM remains unclear. Herein, we report a high-speed FPM technique based on programmable annular illuminations (AIFPM). The optical-transfer-function (OTF) analysis of FPM reveals that the low-frequency phase information can only be correctly recovered if the LEDs are precisely located at the edge of the objective numerical aperture (NA) in the frequency space. By using only 4 low-resolution images corresponding to 4 tilted illuminations matching a 10×, 0.4 NA objective, we present the high-speed imaging results of in vitro Hela cells mitosis and apoptosis at a frame rate of 25 Hz with a full-pitch resolution of 655 nm at a wavelength of 525 nm (effective NA = 0.8) across a wide field-of-view (FOV) of 1.77 mm2, corresponding to a space–bandwidth–time product of 411 megapixels per second. Our work reveals an important capability of FPM towards high-speed high-throughput imaging of in vitro live cells, achieving video-rate QPI performance across a wide range of scales, both spatial and temporal.

Fourier ptychographic microscopy (FPM) is an attractive method to extend the resolution beyond the conventional limit defined by a microscope optics, sharing properties with ptychographic, synthetic aperture imaging and phase retrieval. The algorithm uses a sequence of low-resolution (LR) images acquired under angularly varying illumination to reconstruct a high-resolution (HR) image. However, traditional FPM may trap to the sub-optimal solution, since brute-force replacement in the Fourier domain is applied. To address this problem, we propose here a weighted replacement for Fourier ptychographic microscopy (WFPM). We employ the weighted average of spectrums corresponding to different illumination angle to replace the overlapped regions in the Fourier domain. A series of experimental results demonstrate that the reconstructed image using the proposed WFPM shows a better quality and a faster convergence compared with the results obtained by FPM.

Fourier ptychographic microscopy (FPM) emerged as a prominent imaging technique in 2013, attracting significant interest due to its remarkable features such as precise phase retrieval, expansive field of view (FOV), and superior resolution. Over the past decade, FPM has become an essential tool in microscopy, with applications in metrology, scientific research, biomedicine, and inspection. This achievement arises from its ability to effectively address the persistent challenge of achieving a trade-off between FOV and resolution in imaging systems. It has a wide range of applications, including label-free imaging, drug screening, and digital pathology. In this comprehensive review, we present a concise overview of the fundamental principles of FPM and compare it with similar imaging techniques. In addition, we present a study on achieving colorization of restored photographs and enhancing the speed of FPM. Subsequently, we showcase several FPM applications utilizing the previously described technologies, with a specific focus on digital pathology, drug screening, and three-dimensional imaging. We thoroughly examine the benefits and challenges associated with integrating deep learning and FPM. To summarize, we express our own viewpoints on the technological progress of FPM and explore prospective avenues for its future developments.

White blood cells (WBC) play a significant role in human immune system, so WBC detection is a meaningful work. In this paper, we propose a novel framework which combines Fourier ptychographic microscopy (FPM) and SO-you only look once (YOLO) for WCB detection. FPM is a recently developed microscope technology which can achieve high-resolution, wide-field, and quantitative phase imaging at the same time without mechanical moving and phase measurement. With the FPM, we can get high resolution, wide field-of-view blood cell images at one time. After obtaining high-resolution images, we propose SO-YOLO as our detection method. As a convolutional neural network, the SO-YOLO outperforms the state-of-the-art detection methods both in accuracy and speed. The proposed framework in this paper can be used in clinical diagnose, because the experiment setup is flexible and the detection is fast. The data set in this paper is made with FPM by ourselves. Our data set contains 1000 color high resolution images and all the WBC has been detected and counted by a human expert. The detection results by the experts are regarded as the ground truth. The experiment results show that the SO-YOLO performs well in detecting small objects compared with other methods.

Phase retrieval is an important tool to unveil wavefront of light, especially in high performance microscopy such as Fourier ptychographic microscopy (FPM). In general phase-retrieval methods, the resolution and the number of measurements are in a trade-off relationship. Inspired by FPM, we devise what we believe is a novel microscopic phase-retrieval method, termed single-shot FPM (SSFPM). In our approach, the imaging performance exceeds the trade-off relationship in that it enables phase retrieval for high resolution with a single measurement. By placing the lens array at the Fourier plane of the objective lens, multiple intensity profiles required for the FPM algorithm are collected in a single shot. To achieve enough redundancy of data for satisfying convergence condition of FPM, the specimen is simultaneously illuminated by multiple light-emitting diodes. SSFPM reconstructs quantitative phase profile and enhances the resolution sacrificed by applying lens-array imaging. We demonstrate the performance of SSFPM with numerical simulation and experiments. The prototype achieves lateral resolution of 3.10 μm over a field of view of 0.34 mm2. Without an interferometer or scanning devices, SSFPM can reconstruct high resolution of a complex profile with a single shot.

Fourier ptychographic microscopy (FPM) is a powerful computational imaging technique that enables the reconstruction of high-resolution, wide field-of-view complex images by fusing multiple low-resolution intensity measurements obtained under varying illumination angles. In practice, FPM remains sensitive to calibration and, in particular, to the initialization of the pupil function in the non-convex joint recovery of object and pupil. We introduce pupil-initialized FPM (PI-FPM), a physically informed pupil-phase initialization computed directly from standard FPM measurements prior to reconstruction. PI-FPM exploits the relationship between lateral shifts in brightfield images and local pupil-phase gradients, and estimates low-order aberrations via a Zernike least-squares fit. The method is hardware-agnostic, algorithm-independent, and requires no additional calibration or specialized acquisition, enabling retroactive use on existing datasets. Across synthetic and real experiments, PI-FPM improves convergence stability and reconstruction fidelity, with gains up to 8 dB PSNR and robust performance under strong aberrations (up to 2 π radians root mean square (RMS) wavefront error), particularly in outer field regions. As a result, PI-FPM reduces the reliance on expert calibration and significantly enhances the practical applicability of FPM for routine use in real-world imaging scenarios, including large-field whole-slide imaging where outer-field robustness is critical.

The integration of neural network and physical reconstruction model for Fourier ptychographic microscopy

Fourier ptychographic microscopy (FPM) is an emerging computational imaging technique that breaks through the optical diffraction limitations, enabling large‐field, high‐resolution observation of biological samples. However, traditional FPM iterative reconstruction algorithms suffer from high computational complexity and poor real‐time performance. Additionally, existing data‐driven FPM models exhibit limited generalization due to simulation‐dominated training data. To overcome these challenges, the FPM‐BioCell dataset of multi‐source biological samples is constructed based on the previously proposed forward wavelet‐transform model family (WL‐FPM). Furthermore, a generative adversarial network integrating wavelet transform blocks and mamba‐inspired linear attention models, termed WM‐FPM, is proposed for FPM reconstruction. Through multi‐scale feature extraction, long sequence modeling, and adversarial training, WM‐FPM breaks the speed‐accuracy trade‐off in FPM reconstruction. It outperforms state‐of‐the‐art deep learning models on the FPM‐BioCell dataset with SSIM (0.71), MS‐SSIM (0.87), PSNR (28.55), FID (35.35), and LPIPS (0.27). Crucially, when validated against an independent optical ground truth, WM‐FPM not only generalizes effectively to unseen samples but also achieves a speedup of over 100× compared to the physics‐based WL‐FPM, reconstructing a full‐field 12288 × 12288‐pixel image in merely 5.63 s from a single 2048 × 2048‐pixel input, without compromising fidelity. The code and dataset are open‐sourced to facilitate the rapid deployment of FPM in biomedicine.

Fourier ptychographic microscopy (FPM) is a recently proposed computational imaging method that achieves both high resolution (HR) and wide field of view. In the conventional FPM model, the sample is assumed to be a 2D thin layer, and a series of low-resolution images at different illumination angles is used for HR image reconstruction. However, in practice, the sample complex amplitude distribution is usually mixed with some unknown background interferences. These background interferences may result from inhomogeneous media distribution or other defocusing layers, etc. The background interference will significantly degrade FPM reconstruction results, but so far it is not considered in the conventional FPM algorithm. Here, we propose a method that adaptively separates background interferences for each illumination angle. Experimental results show that the proposed method has a faster convergence speed and better reconstruction accuracy than the conventional FPM model.

Fourier ptychographic microscopy (FPM) imaging is a computational imaging technology that can reconstruct wide-field high-resolution (HR) images. It uses a series of low-resolution images captured by a camera under different illumination angles. The images are stitched in the Fourier domain to expand their spectral range. Under high-angle illumination, a dark-field image is noisy with a low signal-to-noise ratio, which significantly reduces the reconstruction quality of FPM. Conventional reconstruction algorithms often have low FPM imaging performance and efficiency due to optimization strategies. In response to these problems, this paper proposes an FPM imaging method based on an improved phase recovery strategy to optimize the alternating iterative algorithm. The technique uses an improved threshold method to reduce noise in the image preprocessing stage to maximize the retention of high-frequency sample information. Moreover, an adaptive control factor is added in the subsequent iterative update process to balance the sample spectrum function. This study verifies the effectiveness of the proposed method on both simulation and experimental images. The results show that the proposed method can effectively suppress image background noise and has a faster convergence speed and higher robustness. In addition, it can be used to reconstruct HR complex amplitude images of objects under wide field-of-view conditions.

Structural imaging using light microscopy is a cornerstone of histology and cytology. However, the utility of the optical microscope for diagnostic imaging is limited by the fundamental tradeoff between the field of view and spatial resolution and a reliance on exogenous dyes to generate sufficient image contrast. Fourier Ptychographic Microscopy (FPM) is a complex imaging modality with the potential to overcome these limitations by recovering high-resolution images of sample amplitude and phase from a set of low-resolution raw images captured under inclined illumination. In this article we explore the application of FPM to clinical imaging using a simple, low-cost FPM system and simulated and experimental data to explore the influence of both image acquisition parameters and hardware configuration on image quality and imaging throughput. The practical performance of the method is investigated by imaging peripheral blood films and histological tissue sections. We find that, at the cost of increased computational complexity, FPM increases the information capture capacity of the optical microscope significantly, allowing label-free examination and quantification of features such as tissue and cell morphology over large sample areas.

We propose a physics-assisted deep neural network scheme in Fourier ptychographic microscopy (FPM) using untrained deep neural network priors (FPMUP) to achieve a high-resolution image reconstruction from multiple low-resolution images. Unlike the traditional training type of deep neural network that requires a large labelled dataset, this proposed scheme does not require training and instead outputs the high-resolution image by optimizing the parameters of neural networks to fit the experimentally measured low-resolution images. Besides the amplitude and phase of the sample function, another two parallel neural networks that generate the general pupil function and illumination intensity factors are incorporated into the carefully designed neural networks, which effectively improves the image quality and robustness when both the aberration and illumination intensity fluctuation are present in FPM. Reconstructions using simulated and experimental datasets are demonstrated, showing that the FPMUP scheme has better image quality than the traditional iterative algorithms, especially for the phase recovery, but at the expense of increasing computational cost. Most importantly, it is found that the FPMUP scheme can predict the Fourier spectrum of the sample outside synthetic aperture of FPM and thus eliminate the ringing effect of the recovered images due to the spectral truncation. Inspired by deep image prior in the field of image processing, we may impute the expansion of Fourier spectrums to the deep prior rooted in the architecture of the careful designed four parallel deep neural networks. We envisage that the resolution of FPM will be further enhanced if the Fourier spectrum of the sample outside the synthetic aperture of FPM is accurately predicted.

Fourier ptychographic microscopy (FPM) is a computational imaging technique developed in recent years. The validity of the final reconstruction results is closely related to many systematic errors, among which the pupil aberration is a key one. Although the traditional FPM can recover the pupil function in the iterative process, it increases the dependence on data redundancy invisibly, and cannot satisfy the high-speed and low redundancy FPM reconstruction with few images. In this paper, a pupil function recovering method based on annular illumination is proposed, which only uses a small number of LEDs matching the NAobj for illumination. In addition, the Zernike polynomial is used to constrain the freedom degree of aberration, to ensure that the object and pupil can be recovered correctly with fewer data. The simulation results show that the proposed method can recover and correct the pupil function within 6 annular bright-field images, which greatly reduces the requirement of data redundancy and provides the possibility for high-speed aberration recovery based on FPM.

As a typical computational method, Fourier ptychographic microscopy (FPM) can realize high spatial resolution and quantitative phase imaging while preserving the large field of view with a low numerical aperture (NA) objective. A programmable light-emitting diode (LED) array is used as a typical illuminator in an FPM system, and the illumination parameters of each LED element are crucial to the success of the FPM reconstruction algorithm. Compared with LED arrays arranged in rectangular arrays, LED arrays with special structures such as domes or rings can effectively improve FPM imaging results and imaging efficiency. As a trade-off, their calibration difficulty is greatly increased due to the lack of geometric constraints of rectangular arrays. In this paper, we propose an effective hybrid full-pose parameter calibration method for freeform LED array illuminators, combining stereoscopic 3D imaging techniques and the geometric constraints of the microscopic platform. First, a stereovision system is used to obtain the accurate 3D position of each LED element of the freeform illuminator and to construct a rigid 3D coordinate LED array system. Then, calibration between the coordinate system of the LED array and that of the optical imaging component is realized according to the geometric features of the brightfield-to-darkfield edges. Finally, we verify the feasibility and effectiveness of the proposed method through full-pose parameter calibration of LED arrays with different arrangement rules.