High-fidelity, model-driven deep learning network for phase-only computer-generated holography (Conference Presentation)

We report a novel and non-iterative method for the generation of phase-only Fourier hologram for image projection. Briefly, target image is first added with a special quadratic phase and then padded with zeros. A complex Fourier hologram is generated via the simple fast Fourier transform. Subsequently, the error diffusion algorithm is applied to convert the complex hologram into a phase-only hologram. The numerical, as well as the optical reconstructed images with the proposed method are of higher visual quality and contain less speckle noise compared to the original random phase method, which add the random phase to the target image and then preserve the phase component of the complex hologram. The influences of quadratic phase and zero-padding on the image quality are also discussed in detail.

We suggest the use of halftone images for the generation of phase-only Fresnel holograms. The hologram is calculated by spatial filtering in the frequency plain of the two-dimensional cosine Transform. The phase-only hologram is obtained by keeping in the hologram plane only the phase distribution of the inverse light propagation of the desired halftone image located in the reconstruction plane. No iterative method is used in the calculation.

HoloGen: An open-source toolbox for high-speed hologram generation

Large field-of-view lensless holographic dynamic projection system with uniform illumination and U-net acceleration

When applying phase-only computer-generated hologram (CGH) as a standard model of optical measurement in computer-generated holography for aspheric surface testing, it has the advantage of simplifying optical path configuration and improving the diffraction efficiency of the incoming light. However, errors always exists during the encoding process of fabricating multiphase level CGHs and this kind of errors will be amplified level by level in the measurement. According to the analysis of the encoding error, the error of CGH increases linearly when its quantified period increases. For example, if the quantified period is 32, the maximum of encoding error is 16.46 which can lead wave-front aberration 0.085λ of a secondary parabolic surface with 512×512 sampling pixels. In this article, an optimization method based on deviation of minimum boundary value has been used to eliminate the encoding error of CGH. In the experiment, we use a liquid crystal spatial light modulator to generate CGHs and measure residuals of reconstructed wave-front of a secondary parabolic surface. The measurement results show that average decrease of the RMS values of the residuals is 0.07 when their periods range from 3 to 6, which indicates the optimization method is effective.

Visual perception of noise in a simulated holographic display—A user study

The higher-order Hermite–Gaussian (HG) modes exhibit complex spatial distributions and find a wide range of applications in fields such as quantum information processing, optical communications, and precision measurements. In recent years, the advancement of deep learning has emerged as an effective approach for generating higher-order HG modes. However, the traditional data-driven deep learning method necessitates a substantial amount of labeled data for training, entails a lengthy data acquisition process, and imposes stringent requirements on system stability. In practical applications, these methods are confronted with challenges such as the high cost of data labeling. This paper proposes a method that integrates a physical model with deep learning. By utilizing only a single intensity distribution of the target optical field and incorporating the physical model, the training of the neural network can be accomplished, thereby eliminating the dependency of traditional data-driven deep learning methods on large datasets. Experimental results demonstrate that, compared with the traditional data-driven deep learning method, the method proposed in this paper yields a smaller root mean squared error between the generated higher-order HG modes. The quality of the generated modes is higher, while the training time of the neural network is shorter, indicating greater efficiency. By incorporating the physical model into deep learning, this approach overcomes the limitations of traditional deep learning methods, offering a novel solution for applying deep learning in light field manipulation, quantum physics, and other related fields.

This article firstly provides an intuitive perspective of understanding the traditional Gerchberg-Saxton (GS) algorithm, then quantitatively demonstrates the effectiveness of the konoform operation which is not uncommon in multiplexing holography and related fields. We conclude the first step of GS algorithm not only plays a vital role in successfully reconstructing target images, but also helps to justify the kinoform operation when superposing two phase-only holograms. A quantitative method to calculate the signal-to-noise ratio (SNR) of multiplexing holograms after kinoform operation is also proposed in this article.

Recently, there is an increasing trend of adopting digital holography in practical applications such as projectors, head-up display (HUD), and near-eye augmented reality (AR) devices. These demands have led to numerous research works in the past decade, focusing on developing low cost and compact holographic display system. Amongst these research works, investigation on generation of phase-only holograms has become an area of immense interest. A phase-only hologram (POH) has two major advantages. First it can be displayed with a single phase-only SLM. Second, the reconstructed image is brighter, and free from the de-focused twin image. In this tutorial, a number of methods for the generation of POH, will be reviewed. Emphasis will be placed on methods that are non-iterative and computationally efficient so that they can be applied to generate phase-only hologram at high frame-rate. Moreover, these works also have the advantage that the POHs generated can be displayed directly on the phase-only SLM with little or no optical accessories, hence simplifying the complexity and cost of the optical setup.

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

A tractor beam, which has the ability to attract objects, is a class of special optical beams. Currently, people are using the holographic technology to shape complex optical tractor beams for both fundamental research and practical applications. However, most of the work reported is focusing on generating two-dimensional (2D) tractor beams and simple three-dimensional (3D) tractor beams, which has limitations in the further development on mechanism and application of beam shaping. In the present work, we are introducing our study in designing multiple 3D tractor beams with spatial location regulated independently. Meanwhile, each individual beam could be prescribed along arbitrary geometric curve and twisted at arbitrary angles as desired. In our method, the computer-generated hologram (CGH) of each curve is calculated, and all the CGHs are multiplexed and encoded into one phase-only hologram by adding respective linear phase grating such that different 3D curves appeared in the different positions of the focal regions. We experimentally prove that the generation of optical tractor beams at 3D configuration can be readily achieved. The generated beams in the present study are especially useful for applications such as multiple micro-machining optical trapping and complex 3D manipulation.

The combination of computer‐generated holography (CGH) and deep learning has opened the possibility to generate both realtime and high‐quality holograms. However, the widely‐used data‐driven deep learning method faces the limitation of the labeled training datasets. To address the above challenges, we propose model‐driven neural networks for high‐fidelity and high‐speed phase‐only hologram (POH) generation. The Fresnel diffraction model is introduced as the constraint to train the networks. Furthermore, the improvements of the network architecture and diffraction model effectively improve the reconstruction quality. High‐fidelity full‐color optical reconstructions of the 4K holograms have been achieved.

Objective. Neural decoding is an important tool in neural engineering and neural data analysis. Of various machine learning algorithms adopted for neural decoding, the recently introduced deep learning is promising to excel. Therefore, we sought to apply deep learning to decode movement trajectories from the activity of motor cortical neurons. Approach. In this paper, we assessed the performance of deep learning methods in three different decoding schemes, concurrent, time-delay, and spatiotemporal. In the concurrent decoding scheme where the input to the network is the neural activity coincidental to the movement, deep learning networks including artificial neural network (ANN) and long-short term memory (LSTM) were applied to decode movement and compared with traditional machine learning algorithms. Both ANN and LSTM were further evaluated in the time-delay decoding scheme in which temporal delays are allowed between neural signals and movements. Lastly, in the spatiotemporal decoding scheme, we trained convolutional neural network (CNN) to extract movement information from images representing the spatial arrangement of neurons, their activity, and connectomes (i.e. the relative strengths of connectivity between neurons) and combined CNN and ANN to develop a hybrid spatiotemporal network. To reveal the input features of the CNN in the hybrid network that deep learning discovered for movement decoding, we performed a sensitivity analysis and identified specific regions in the spatial domain. Main results. Deep learning networks (ANN and LSTM) outperformed traditional machine learning algorithms in the concurrent decoding scheme. The results of ANN and LSTM in the time-delay decoding scheme showed that including neural data from time points preceding movement enabled decoders to perform more robustly when the temporal relationship between the neural activity and movement dynamically changes over time. In the spatiotemporal decoding scheme, the hybrid spatiotemporal network containing the concurrent ANN decoder outperformed single-network concurrent decoders. Significance. Taken together, our study demonstrates that deep learning could become a robust and effective method for the neural decoding of behavior.

Knowledge-driven building extraction method exhibits a restricted adaptability scope and is vulnerable to external factors that affect its extraction accuracy. On the other hand, data-driven building extraction method lacks interpretability, heavily relies on extensive training data, and may result in extraction outcomes with building boundary blur issues. The integration of pre-existing knowledge with data-driven learning is essential for the intelligent identification and extraction of buildings from high-resolution aerial images. To overcome the limitations of current deep learning building extraction networks in effectively leveraging prior knowledge of aerial images, a geometric significance-aware deep mutual learning network (GSDMLNet) is proposed. Firstly, the GeoSay algorithm is utilized to derive building geometric significance feature maps as prior knowledge and integrate them into the deep learning network to enhance the targeted extraction of building features. Secondly, a bi-directional guidance attention module (BGAM) is developed to facilitate deep mutual learning between the building feature map and the building geometric significance feature map within the dual-branch network. Furthermore, the deployment of an enhanced flow alignment module (FAM++) is utilized to produce high-resolution, robust semantic feature maps with strong interpretability. Ultimately, a multi-objective loss function is crafted to refine the network’s performance. Experimental results demonstrate that the GSDMLNet excels in building extraction tasks within densely populated and diverse urban areas, reducing misidentification of shadow-obscured regions and color-similar terrains lacking building structural features. This approach effectively ensures the precise acquisition of urban building information in aerial images.

Acoustic holography (AH) provides a promising technique for arbitrary acoustic field reconstruction, supporting many applications like robotic micro-nano manipulation, neuromodulation, volumetric imaging, and virtual reality. In AH, three-dimensional (3D) acoustic fields quantified with complex-valued acoustic pressures are reconstructed by virtue of two-dimensional (2D) acoustic holograms. Phase-only hologram (POH) is recently regarded as an energy-efficient way for AH, which is typically implemented by a dynamically programmable phased array of transducers (PATs). As a result, spatiotemporal precise acoustic field reconstruction is enabled by precise, dynamic, and individual actuation of PAT. Thus, 2D POH is required per arbitrary acoustic fields, which can be viewed as a physical inverse problem. However, solving the aforementioned physical inverse problem in numerical manners poses challenges due to its non-linear, high-dimensional, and complex coupling natures. The existing iterative algorithms like the iterative angular spectrum approach (IASA) and iterative backpropagation (IB) still suffer from speed-accuracy trade-offs. Hence, this paper explores a novel physics-iterative-reinforced deep learning method, in which frequency-argument contrastive learning is proposed facilitated by the inherent physical nature of AH, and the energy conservation law is under consideration. The experimental results demonstrate the effectiveness of the proposed method for acoustic field reconstruction, highlighting its significant potential in the domain of acoustics, and pushing forward the combination of physics into deep learning.