High-resolution lensless holographic microscopy using a physics-aware deep network.

An encryption system that combines compressive sensing (CS) and two‐step parallel phase shifting digital holography (PPSDH) using double random phase encoding (DRPE) is presented in this paper. The two‐step PPSDH is a linear inline holographic scheme and is much suitable for encrypting the 2D/3D information in a single exposure. The distribution of random phase mask (RPM) in the DRPE is implemented using circular harmonic key which increases the security of the encryption process. In this system, the keys used to encrypt are spatial positions of the planes, wavelength, and rotation of the circular harmonics in RPMs, and CS acts as an additional key that makes the system more secure than the conventional optical encryption methods. At the transmission end, two‐step PPSDH is applied to encrypt the object information in single hologram. The digital mirror device (DMD) is placed between the object and a single‐pixel detector for acquiring fewer hologram measurements. At the receiver end, the single digital hologram is numerically recovered by using a CS optimization problem. The original complex object field is decrypted from the CS recovered holograms by the inversion of two‐step PPSDH process with the help of the correct keys. The numerical simulations are presented for complex 2D and 3D objects to test the feasibility of the proposed encryption and decryption system. The proposed method carried out intensity and phase reconstruction of the original object field using single‐pixel compressive imaging. The computer simulation results demonstrated that the encrypted information is highly secured with the rotation of the circular harmonic key. The sensitivity of the decrypted intensity and phase images is also studied with variations of the encrypted keys. The obtained results show that the proposed encryption scheme is feasible and has better security performance and robustness.

Deep neural networks have made significant achievements in representation learning of traditionally man-made features, especially in terms of complex objects. Over the decades, this learning process has attracted thousands of researchers and has been widely used in the speech, visual, and text recognition fields. One deep network multi-layer extreme learning machine (ML-ELM) achieves a good performance in representation learning while inheriting the advantages of faster learning and the approximating capability of the extreme learning machine (ELM). However, as with most deep networks, the ML-ELM’s algorithmic performance largely depends on the probability distribution of the training data. In this paper, we propose an improved ML-ELM made via using the local significant regions at the input end to enhance the contributions of these regions according to the idea of the selective attention mechanism. To avoid involving and exploring the complex principle of the attention system and to focus on the clarification of our local regional enhancement idea, the paper only selects two typical attention regions. One is the geometric central region, which is normally the important region to attract human attention due to the focal attention mechanism. The other is the task-driven interest region, with facial recognition as an example. The comprehensive experiments are done on the three public datasets of MNIST, NORB, and ORL. The comparison experiment results demonstrate that our proposed region-enhanced ML-ELM (RE-ML-ELM) achieves performance increases in important feature learning by utilizing the apriori knowledge of attention and has a higher recognition rate than that of the normal ML-ELM and the basic ELM. Moreover, it benefits from the non-iterative parameter training method of other ELMs, and our proposed algorithm outperforms most state-of-the-art deep networks such as deep belief network(DBN), in the aspects of training efficiency. Furthermore, because of the deep structure with fewer hidden nodes at each layer, our proposed RE-ML-ELM achieves a comparable training efficiency to that of the ML-ELM but has a higher training speed with the basic ELM, which is normally the width single network that has more hidden nodes to obtain the similar recognition accuracy with the deep networks. Based on our idea of combining the apriori knowledge of the human selective attention system with the data learning, our proposed region-enhanced ML-ELM increases the image classification performance. We believe that the idea of intentionally combining psychological knowledge with the most algorithms based on data-driven learning has the potential to improve their cognitive computing ability.

Fourier ptychographic microscopy (FPM) is a newly developed microscopic technique for large field of view, high-resolution and quantitative phase imaging by combining the techniques from ptychographic imaging, aperture synthesizing and phase retrieval. In FPM, an LED array is utilized to illuminate the specimen from different angles and the corresponding intensity images are synthesized to reconstruct a high-resolution complex field. As a flexible and low-cost approach to achieve high-resolution, wide-field and quantitative phase imaging, FPM is of enormous potential in biomedical applications such as hematology and pathology. Conventionally, the FPM reconstruction problem is solved by using a phase retrieval method, termed Alternate Projection. By iteratively updating the Fourier spectrum with low-resolution-intensity images, the result converges to a high-resolution complex field. Here we propose a new FPM reconstruction framework with deep learning methods and design a multiscale, deep residual neural network for FPM reconstruction. We employ the widely used open-source deep learning library PyTorch to train and test our model and carefully choose the hyperparameters of our model. To train and analyze our model, we build a large-scale simulation dataset with an FPM imaging model and an actual dataset captured with an FPM system. The simulation dataset and actual dataset are separated as training datasets and test datasets, respectively. Our model is trained with the simulation training dataset and fine tuned with the fine-tune dataset, which contains actual training data. Both our model and the conventional method are tested on the simulation test dataset and the actual test dataset to evaluate the performances. We also show the reconstruction result of another neural network-based method for comparison. The experiments demonstrate that our model achieves better reconstruction results and consumes much less time than conventional methods. The results also point out that our model is more robust under system aberrations such as noise and blurring (fewer intensity images) compared with conventional methods.

Measurement of refractive index change induced by dark reaction of photopolymer with digital holographic quantitative phase microscopy

The optical monitoring of multiple single neuron activities requires high-throughput parallel acquisition of signals at millisecond temporal resolution. To this aim, holographic two-photon microscopy (2PM) based on spatial light modulators (SLMs) has been developed in combination with standard laser scanning microscopes. This requires complex coordinate transformations for the generation of holographic patterns illuminating the points of interest. We present a simpler and fully digital setup (SLM-2PM) which collects three-dimensional two-photon images by only exploiting the SLM. This configuration leads to an accurate placement of laser beamlets over small focal volumes, eliminating mechanically moving parts and making the system stable over long acquisition times. Fluorescence signals are diffraction limited and are acquired through a pixelated detector, setting the actual limit to the acquisition rate. High-resolution structural images were acquired by raster-scanning the sample with a regular grid of excitation focal volumes. These images allowed the selection of the structures to be further investigated through an interactive operator-guided selection process. Functional signals were collected by illuminating all the preselected points with a single hologram. This process is exemplified for high-speed (up to 1kHz) two-photon calcium imaging on acute cerebellar slices.

Fourier phase recovery techniques focus on how to reconstruct object information from phaseless measurement. Generally, such model-based phase recovery algorithms are difficult to obtain high-quality reconstructions in the presence of noise interference. Hence, we proposed a phase retrieval algorithm with deep denoiser networks. Firstly, an optimization model is constructed for the phase retrieval problem, then the alternating direction method of multipliers method is used to solve optimization problem iteratively. Besides, a well-trained deep neural network act as plug-and-play denoiser to participate the process of algorithm. Our method combines the model information of traditional phase retrieval algorithm and the fitting ability of the deep neural network, experiments show that it can achieve higher reconstruction result in the face of noisy image, and the generalization ability is also improved compared to end-to-end method.

We present a deep learning-enabled holographic polarization microscope that only requires one polarization state to image/quantify birefringent specimen. This framework reconstructs quantitative birefringence retardance and orientation images from the amplitude/phase information obtained using a lensless holographic microscope with a pair of polarizer and analyzer. We tested this technique with various birefringent samples including monosodium urate and triamcinolone acetonide crystals to demonstrate that the deep network can accurately reconstruct the retardance and orientation image channels. This method has a simple optical design and presents a large field-of-view (>20-30mm2), which might broaden the access to advanced polarization microscopy techniques in low-resource-settings.

Cervical cancer is one of the most common types of cancer among women, which has higher death-rate than many other cancer types. The most common way to diagnose cervical cancer is to analyze images of cervical cells, which is performed using Pap smear imaging test. Early and accurate diagnosis can save the lives of many patients and increase the chance of success of treatment methods. Until now, various methods have been proposed to diagnose cervical cancer based on the analysis of Pap smear images. Most of the existing methods can be divided into two groups of methods based on deep learning techniques or machine learning algorithms. In this study, a combination method is presented, whose overall structure is based on a machine learning strategy, where the feature extraction stage is completely separate from the classification stage. However, in the feature extraction stage, deep networks are used. In this paper, a multi-layer perceptron (MLP) neural network fed with deep features is presented. The number of hidden layer neurons is tuned based on four innovative ideas. Additionally, ResNet-34, ResNet-50 and VGG-19 deep networks have been used to feed MLP. In the presented method, the layers related to the classification phase are removed in these two CNN networks, and the outputs feed the MLP after passing through a flatten layer. In order to improve performance, both CNNs are trained on related images using the Adam optimizer. The proposed method has been evaluated on the Herlev benchmark database and has provided 99.23 percent accuracy for the two-classes case and 97.65 percent accuracy for the 7-classes case. The results have shown that the presented method has provided higher accuracy than the baseline networks and many existing methods.

The relation between serum and red blood cell (RBC) inorganic phosphate levels, RBC 2,3-diphosphoglycerate (2,3-DPG) levels, RBC nucleotide phosphate (Pn), and RBC total phosphate (Pt) levels were studied during the early phases of treatment and recovery from diabetic ketoacidosis (DKA). A steady drop in serum inorganic phosphate was found during the first 24 hours of insulin treatment and was most profound at 24 hours. No statistically significant changes (P less than 0.05) were found in red cell inorganic phosphate or nucleotide phosphate levels during the 24-hour study period. The levels of total red cell phosphate were lower in this group of patients than in nonacidotic diabetic subjects and decreased slightly after 24 hours of treatment. The red cell 2,3-DPG levels were low at the initiation of therapy and remained low during the 24-hour study period. Glucose, bicarbonate, lactate, and ketone levels fell in linear patterns with treatment. In view of the current evidence for the effects of low 2,3-DPG on oxygen delivery and the relation of low serum phosphate levels to RBC glycolysis and 2,3-DPG formation, this study reemphasizes the need for phosphate replacement during the early phases of treatment of DKA.

Red blood cell (RBC) phase images that are numerically reconstructed by digital holographic microscopy (DHM) can describe the cell structure and dynamics information beneficial for a quantitative analysis of RBCs. However, RBCs investigated with time-lapse DHM undergo temporal displacements when their membranes are loosely attached to the substrate during sedimentation on a glass surface or due to the microscope drift. Therefore, we need to develop a tracking algorithm to localize the same RBC among RBC image sequences and dynamically monitor its biophysical cell parameters; this information is helpful for studies on RBC-related diseases and drug tests. Here, we propose a method, which is a combination of the mean-shift algorithm and Kalman filter, to track a single RBC and demonstrate that the optical path length of the single RBC can be continually extracted from the tracked RBC. The Kalman filter is utilized to predict the target RBC position in the next frame. Then, the mean-shift algorithm starts execution from the predicted location, and a robust kernel, which is adaptive to changes in the RBC scale, shape, and direction, is designed to improve the accuracy of the tracking. Finally, the tracked RBC is segmented and parameters such as the RBC location are extracted to update the Kalman filter and the kernel function for mean-shift tracking; the characteristics of the target RBC are dynamically observed. Experimental results show the feasibility of the proposed algorithm.

Counting morphologically normal cells in human red blood cells (RBCs) is extremely beneficial in the health care field. We propose a three-dimensional (3-D) classification method of automatically determining the morphologically normal RBCs in the phase image of multiple human RBCs that are obtained by off-axis digital holographic microscopy (DHM). The RBC holograms are first recorded by DHM, and then the phase images of multiple RBCs are reconstructed by a computational numerical algorithm. To design the classifier, the three typical RBC shapes, which are stomatocyte, discocyte, and echinocyte, are used for training and testing. Nonmain or abnormal RBC shapes different from the three normal shapes are defined as the fourth category. Ten features, including projected surface area, average phase value, mean corpuscular hemoglobin, perimeter, mean corpuscular hemoglobin surface density, circularity, mean phase of center part, sphericity coefficient, elongation, and pallor, are extracted from each RBC after segmenting the reconstructed phase images by using a watershed transform algorithm. Moreover, four additional properties, such as projected surface area, perimeter, average phase value, and elongation, are measured from the inner part of each cell, which can give significant information beyond the previous 10 features for the separation of the RBC groups; these are verified in the experiment by the statistical method of Hotelling's T-quare test. We also apply the principal component analysis algorithm to reduce the dimension number of variables and establish the Gaussian mixture densities using the projected data with the first eight principal components. Consequently, the Gaussian mixtures are used to design the discriminant functions based on Bayesian decision theory. To improve the performance of the Bayes classifier and the accuracy of estimation of its error rate, the leaving-one-out technique is applied. Experimental results show that the proposed method can yield good results for calculating the percentage of each typical normal RBC shape in a reconstructed phase image of multiple RBCs that will be favorable to the analysis of RBC-related diseases. In addition, we show that the discrimination performance for the counting of normal shapes of RBCs can be improved by using 3-D features of an RBC.

This study aimed to evaluate the protective efficacy of a novel red blood cell (RBC) preservation solution on erythrocytes during storage, with specific focus on RBCs from Type 2 diabetes mellitus (T2DM) patients. Whole blood samples were obtained from T2DM patients and healthy donors. RBCs were isolated and stored in either the novel solution or the conventional solution. Key parameters including RBC concentration, mean corpuscular volume (MCV), red cell distribution width (RDW), and lactic acid (LA) levels were analyzed using automated hematology analyzers. Additionally, vesicles were isolated from RBCs and quantified. Cellular oxidative stress indicators, enzyme activities, and cytokines were measured using commercial kits. Furthermore, the structure and cell membrane fluctuations (CMF) of RBCs were observed and assessed using scanning electron microscope and digital holographic microscope. The study revealed that the new storage solution significantly alleviated the reduction in RBC concentration and the increase in MCV, RDW, and LA levels. Additionally, it effectively suppressed the free hemoglobin (FHb) content, hemolysis degree, and hemoglobin A1C (HbA1C) level in stored RBCs. The novel storage solution effectively reduced the accumulation of phosphatidylserine (PS)-exposing RBCs and the formation of vesicles within the RBCs. Furthermore, it markedly mitigated oxidative damage and decreased the proportion of abnormal RBCs in stored samples. The new storage solution played a protective role by maintaining membrane integrity and suppressing oxidative stress in stored RBCs from T2DM patients.

Digital holographic (DH) microscopy is a promising technique for quantitative phase contrast imaging. It provides complex amplitude of the object wavefront, which in turn yields the thickness distribution of the object. An added advantage of the technique is its ability for numerical focusing, which provides the thickness distribution of the object at different axial planes. In this invited paper, we present an overview of our reported work on two beam DH microscopyto acquire different cell parameters for cell imaging and automated cell identification. Applications to automated monitoring of stem cells without destroying the cells and automated identification of malaria infected red blood cells are discussed.

Digital holographic microscope is an ideal tool for quantitative phase contrast imaging of living cells. It yields the thickness distribution of the object under investigation from a single hologram. From a series of holograms the dynamics of the cell under investigation can be obtained. But two-beam digital holographic microscopes has low temporal stability due to uncorrelated phase changes occurring in the reference and object arms. One way to overcome is to use common path techniques, in which, the reference beam is derived from the object beam itself. Both the beams travel along the same path, increasing the temporal stability of the setup. In self-referencing techniques a portion of the object beam is converted into reference beam. It could be achieved by example, using a glass plate to create two laterally sheared versions of the object beam at the sensor, which interfere to produce the holograms/interferograms. This created a common path setup, leading to high temporal stability (~0.6nm). This technique could be used to map cell membrane fluctuations with high temporal stability. Here we provide an overview of our work on the development of temporally stable quantitative phase contrast techniques for dynamic imaging of micro-objects and biological specimen including red blood cells.

The ability to sense, track, identify, and monitor biological micro/nanoorganisms in a real-time, automated, and integrated system is of great importance from both scientific and technological standpoints. Such a system and its possible variants would have numerous applications in a wide spectrum of fields, including defense against biological warfare, disease control, environmental health and safety, and medical treatments. In this paper, we review a comprehensive mixture of optical and computational tools developed in our group aiming at real-time sensing and recognition of biological microorganisms. Digital in-line holographic microscopy is used with both coherent and partially coherent illumination to probe the specimen interferometrically. The interference pattern is then recorded on an optoelectronic image sensor and transferred to a computer where special statistical algorithms are performed to segment, recognize, and track the microorganisms within the field of view of the microscope. The advantages of proposed holographic sensing are described compared to conventional two-dimensional imaging systems. In addition, the theoretical aspects and fundamental limitations of digital in-line holographic microscopy are discussed, which determine the relationship between system parameters and achievable performance. The proposed optical-digital integrated system for automated, real-time sensing and recognition of biological microorganisms has been deemed promising with the potential of widespread application. We demonstrate how the proposed techniques function together in a series of experiments.