54.2: High‐fidelity model‐driven deep learning network for phase‐only computer‐generated holography

The combination of Computer-Generated Holography (CGH) and deep learning has opened the possibility to generate both real-time and high-quality holograms. However, the widely-used data-driven deep learning method faces the problem of the large number of labeled training datasets generated by traditional algorithms, such as Gerchberg–Saxton (GS) iterative algorithm. It always takes a long time and limits the training performance of the network. In this work, we propose a model-driven neural network for high-fidelity Phase-Only Hologram (POH) generation. The Fresnel diffraction process is introduced as the physical model, which makes the network can automatically learn the latent encodings of POHs in an unsupervised way. Furthermore, the sub-pixel convolution upsampling method effectively improves the reconstruction quality. Once the training is completed, the POH of any two-dimensional image can be quickly generated. The calculation time is one to two orders of magnitude faster.

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

In this paper, a fast hologram generation method based on optimal segmentation of sub-computer-generated hologram is proposed. When using the point light source (PLS) model for computer-generated hologram (CGH), the 3D object is discretized into individual object points, which are assumed to independent ideal sources. Then, the sub-CGH can be obtained by calculating the diffraction light field distribution of each ideal PLS, and the final hologram can be generated by further calculating of the sub-CGHs. In the proposed method, the contribution of any pixel in the sub-CGH generated by each ideal PLS to the final reconstructed image is calculated, and each sub-CGH is divided into the optimized diffraction areas (ODAs) and the invalid diffraction areas. Then, only the ODAs of each sub-CGH is pre-calculated and saved for further generating the final hologram. Because of the size of the ODAs are much smaller than that of the sub- CGHs, the hologram generation speed is greatly improved. The proposed method can be used to optimize most of the PLS model-based hologram generation methods. The proposed method is used to optimize the novel look-at-table (NLUT) method and wavefront recording plane (WRP) method, and both of them achieve good acceleration effect. In the optimization of the NLUT method, the sub-CGH database of the traditional NLUT method is replaced by the ODAs database. The final hologram can be generated by calling and superimposing all the ODAs. In the optimization of the WRP method, the WRP is calculated by using the ODAs instead of the sub-CGHs. Then, the final hologram is generated by using the Fresnel diffraction from the WRP to the holographic plane. With the proposed method, the calculation speed of the NLUT method and the WRP method can be significantly improved and the quality of the reconstructed image is not affected.

Knowledge-driven building extraction method exhibits a restricted adaptability scope and is vulnerable to external factors that affect its extraction accuracy. On the other hand, data-driven building extraction method lacks interpretability, heavily relies on extensive training data, and may result in extraction outcomes with building boundary blur issues. The integration of pre-existing knowledge with data-driven learning is essential for the intelligent identification and extraction of buildings from high-resolution aerial images. To overcome the limitations of current deep learning building extraction networks in effectively leveraging prior knowledge of aerial images, a geometric significance-aware deep mutual learning network (GSDMLNet) is proposed. Firstly, the GeoSay algorithm is utilized to derive building geometric significance feature maps as prior knowledge and integrate them into the deep learning network to enhance the targeted extraction of building features. Secondly, a bi-directional guidance attention module (BGAM) is developed to facilitate deep mutual learning between the building feature map and the building geometric significance feature map within the dual-branch network. Furthermore, the deployment of an enhanced flow alignment module (FAM++) is utilized to produce high-resolution, robust semantic feature maps with strong interpretability. Ultimately, a multi-objective loss function is crafted to refine the network’s performance. Experimental results demonstrate that the GSDMLNet excels in building extraction tasks within densely populated and diverse urban areas, reducing misidentification of shadow-obscured regions and color-similar terrains lacking building structural features. This approach effectively ensures the precise acquisition of urban building information in aerial images.

HoloGen: An open-source toolbox for high-speed hologram generation

Deep learning neural network approach to thermal-wave imaging of damage in solids with application to diffusivity measurements of a green (unsintered) metal powder compact slab

Deep learning is a recently developed feature representation technique for data with complicated structures, which has great potential for soft sensing of industrial processes. However, most deep networks mainly focus on hierarchical feature learning for the raw observed input data. For soft sensor applications, it is important to reduce irrelevant information and extract quality-relevant features from the raw input data for quality prediction. To deal with this problem, a novel deep learning network is proposed for quality-relevant feature representation in this article, which is based on stacked quality-driven autoencoder (SQAE). First, a quality-driven autoencoder (QAE) is designed by exploiting the quality data to guide feature extraction with the constraint that the potential features should largely reconstruct the input layer data and the quality data at the output layer. In this way, quality-relevant features can be captured by QAE. Then, by stacking multiple QAEs to construct the deep SQAE network, SQAE can gradually reduce irrelevant features and learn hierarchical quality-relevant features. Finally, the high-level quality-relevant features can be directly applied for soft sensing of the quality variables. The effectiveness and flexibility of the proposed deep learning model are validated on an industrial debutanizer column process.

To test the hypothesis that deep learning (DL) networks reliably detect pulmonary arterial hypertension (PAH) and provide prognostic information. Consecutive patients with PAH, right ventricular (RV) dilation (without PAH), and normal controls were included. An ensemble of deep convolutional networks incorporating echocardiographic views and estimated RV systolic pressure (RVSP) was trained to detect (invasively confirmed) PAH. In addition, DL-networks were trained to segment cardiac chambers and extracted geometric information throughout the cardiac cycle. The ability of DL parameters to predict all-cause mortality was assessed using Cox-proportional hazard analyses. Overall, 450 PAH patients, 308 patients with RV dilatation (201 with tetralogy of Fallot and 107 with atrial septal defects) and 67 normal controls were included. The DL algorithm achieved an accuracy and sensitivity of detecting PAH on a per patient basis of 97.6 and 100%, respectively. On univariable analysis, automatically determined right atrial area, RV area, RV fractional area change, RV inflow diameter and left ventricular eccentricity index (P < 0.001 for all) were significantly related to mortality. On multivariable analysis DL-based RV fractional area change (P < 0.001) and right atrial area (P = 0.003) emerged as independent predictors of outcome. Statistically, DL parameters were non-inferior to measures obtained manually by expert echocardiographers in predicting prognosis. The study highlights the utility of DL algorithms in detecting PAH on routine echocardiograms irrespective of RV dilatation. The algorithms outperform conventional echocardiographic evaluation and provide prognostic information at expert-level. Therefore, DL methods may allow for improved screening and optimized management of PAH.

Objective. Neural decoding is an important tool in neural engineering and neural data analysis. Of various machine learning algorithms adopted for neural decoding, the recently introduced deep learning is promising to excel. Therefore, we sought to apply deep learning to decode movement trajectories from the activity of motor cortical neurons. Approach. In this paper, we assessed the performance of deep learning methods in three different decoding schemes, concurrent, time-delay, and spatiotemporal. In the concurrent decoding scheme where the input to the network is the neural activity coincidental to the movement, deep learning networks including artificial neural network (ANN) and long-short term memory (LSTM) were applied to decode movement and compared with traditional machine learning algorithms. Both ANN and LSTM were further evaluated in the time-delay decoding scheme in which temporal delays are allowed between neural signals and movements. Lastly, in the spatiotemporal decoding scheme, we trained convolutional neural network (CNN) to extract movement information from images representing the spatial arrangement of neurons, their activity, and connectomes (i.e. the relative strengths of connectivity between neurons) and combined CNN and ANN to develop a hybrid spatiotemporal network. To reveal the input features of the CNN in the hybrid network that deep learning discovered for movement decoding, we performed a sensitivity analysis and identified specific regions in the spatial domain. Main results. Deep learning networks (ANN and LSTM) outperformed traditional machine learning algorithms in the concurrent decoding scheme. The results of ANN and LSTM in the time-delay decoding scheme showed that including neural data from time points preceding movement enabled decoders to perform more robustly when the temporal relationship between the neural activity and movement dynamically changes over time. In the spatiotemporal decoding scheme, the hybrid spatiotemporal network containing the concurrent ANN decoder outperformed single-network concurrent decoders. Significance. Taken together, our study demonstrates that deep learning could become a robust and effective method for the neural decoding of behavior.

Introduction. The significance of machine learning under the conditions of digital transformation of industry, and methods of implementing deep learning to provide the performance of trust management systems are considered. The necessity of using convolutional artificial neural networks for deep machine learning is determined. Various technologies and architectures for the implementation of artificial neural networks are briefly considered; a comparative analysis of their performance is carried out. The work objective is to study the need to develop new approaches to the architecture of computing machines for solving problems of deep machine learning in the trust management system implementation.Materials and Methods. In the context of digital transformation, the use of artificial intelligence reaches a new level. The technical implementation of artificial neural systems with deep machine learning is based on the use of one of three basic technologies: high performance computing (HPC) with parallel data processing, neuromorphic computing (NC), and quantum computing (QC).Results. Implementation models for deep machine learning, basic technologies and architecture of computing machines, as well as requirements for trust assurance in control systems using deep machine learning are analyzed. The problem of shortage of computation power for solving such problems is identified. None of the currently existing technologies can solve the full range of learning and impedance problems. The current level of technology does not provide information security and reliability of neural networks. The practical implementation of trust management systems with deep machine learning based on existing technologies for a significant part of the tasks does not provide a sufficient level of performance.Discussion and Conclusions. The study made it possible to identify the challenge of the computation power shortage for solving problems of deep machine learning. Through the analysis of the requirements for trust management systems, the external challenges of their implementation on the basis of existing technologies, and the need to develop new approaches to the computer architecture are determined.

Deep learning methods have become popular among researchers in the field of fault detection. However, their performance depends on the availability of big datasets. To overcome this problem researchers started applying transfer learning to achieve good performance from small available datasets, by leveraging multiple prediction models over similar machines and working conditions. However, the influence of negative transfer limits their application. Negative transfer among prediction models increases when the environment and working conditions are changing continuously. To overcome the effect of negative transfer, we propose a novel deep transfer learning method, coined deep boosted transfer learning, for wind turbine gearbox fault detection that prevents negative transfer and only focuses on relevant information from the source machine. The proposed method is an instance-based deep transfer learning method that updates the weights of the source and the target machine training samples separately. The weights of different source training samples are gradually decreased to reduce the impact on the final model. The proposed method is verified by the Case Western Reserve University bearing and real field wind farm datasets. The results show that the proposed method ignores negative transfer and achieves higher accuracy compared to standard deep learning and deep transfer learning methods.

Chapter 2 - Computational intelligence in healthcare and biosignal processing

Due to the complexity of medical images, traditional medical image classification methods have been unable to meet the actual application needs. In recent years, the rapid development of deep learning theory has provided a technical approach for solving medical image classification. However, deep learning has the following problems in the application of medical image classification. First, it is impossible to construct a deep learning model with excellent performance according to the characteristics of medical images. Second, the current deep learning network structure and training strategies are less adaptable to medical images. Therefore, this paper first introduces the visual attention mechanism into the deep learning model so that the information can be extracted more effectively according to the problem of medical images, and the reasoning is realized at a finer granularity. It can increase the interpretability of the model. Additionally, to solve the problem of matching the deep learning network structure and training strategy to medical images, this paper will construct a novel multiscale convolutional neural network model that can automatically extract high-level discriminative appearance features from the original image, and the loss function uses the Mahalanobis distance optimization model to obtain a better training strategy, which can improve the robust performance of the network model. The medical image classification task is completed by the above method. Based on the above ideas, this paper proposes a medical classification algorithm based on a visual attention mechanism-multiscale convolutional neural network. The lung nodules and breast cancer images were classified by the method in this paper. The experimental results show that the accuracy of medical image classification in this paper is not only higher than that of traditional machine learning methods but also improved compared with other deep learning methods, and the method has good stability and robustness.

Multivariate analysis approaches provide insights into the identification of phenotype associations in brain connectome data. In recent years, deep learning methods including convolutional neural network (CNN) and graph neural network (GNN), have shifted the development of connectome-wide association studies (CWAS) and made breakthroughs for connectome representation learning by leveraging deep embedded features. However, most existing studies remain limited by potentially ignoring the exploration of region-specific features, which play a key role in distinguishing brain disorders with high intra-class variations, such as autism spectrum disorder (ASD), and attention deficit hyperactivity disorder (ADHD). Here, we propose a multivariate distance-based connectome network (MDCN) that addresses the local specificity problem by efficient parcellation-wise learning, as well as associating population and parcellation dependencies to map individual differences. The approach incorporating an explainable method, parcellation-wise gradient and class activation map (p-GradCAM), is feasible for identifying individual patterns of interest and pinpointing connectome associations with diseases. We demonstrate the utility of our method on two largely aggregated multicenter public datasets by distinguishing ASD and ADHD from healthy controls and assessing their associations with underlying diseases. Extensive experiments have demonstrated the superiority of MDCN in classification and interpretation, where MDCN outperformed competitive state-of-the-art methods and achieved a high proportion of overlap with previous findings. As a CWAS-guided deep learning method, our proposed MDCN framework may narrow the bridge between deep learning and CWAS approaches, and provide new insights for connectome-wide association studies.

In this paper we propose a hardware architecture for high-speed computer-generated hologram generation that significantly reduces the number of memory access times to avoid the bottleneck in the memory access operation. For this, we use three main schemes. The first is pixel-by-pixel calculation, rather than light source-by-source calculation. The second is a parallel calculation scheme extracted by modifying the previous recursive calculation scheme. The last scheme is a fully pipelined calculation scheme and exactly structured timing scheduling, achieved by adjusting the hardware. The proposed hardware is structured to calculate a row of a computer-generated hologram in parallel and each hologram pixel in a row is calculated independently. It consists of and input interface, an initial parameter calculator, hologram pixel calculators, a line buffer, and a memory controller. The implemented hardware to calculate a row of a 1920×1080 computer-generated hologram in parallel uses 168,960 lookup tables, 153,944 registers, and 19,212 digital signal processing blocks in an Altera field programmable gate array environment. It can stably operate at 198 MHz. Because of three schemes, external memory bandwidth is reduced to approximately 1/20,000 of the previous ones at the same calculation speed.