DeepAlign, a 3D alignment method based on regionalized deep learning for Cryo-EM

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

Deep learning neural network approach to thermal-wave imaging of damage in solids with application to diffusivity measurements of a green (unsintered) metal powder compact slab

Detecting subtle signals from small earthquakes triggered by transient stresses from the surface waves of large magnitude earthquakes can contribute to a more general understanding of how earthquakes nucleate and interact with each other. However, searching for signals from such small earthquakes in thousands of seismograms is overwhelming, and discriminating them from a miscellany of noise is challenging. Here, we explore how we can automate the detection of such dynamically triggered earthquakes using a simple, diagnostic signal-to-noise ratio (SNR) threshold as well as a convolutional deep metric learning network. Our analysis shows that the deep learning network was more reliable at detecting small earthquakes than the SNR method.

Accurate segmentation of the retinal microvasculature is a critical step in the quantitative analysis of the retinal circulation, which can be an important marker in evaluating the severity of retinal diseases. As manual segmentation remains the gold standard for segmentation of optical coherence tomography angiography (OCT-A) images, we present a method for automating the segmentation of OCT-A images using deep neural networks (DNNs). Eighty OCT-A images of the foveal region in 12 eyes from 6 healthy volunteers were acquired using a prototype OCT-A system and subsequently manually segmented. The automated segmentation of the blood vessels in the OCT-A images was then performed by classifying each pixel into vessel or nonvessel class using deep convolutional neural networks. When the automated results were compared against the manual segmentation results, a maximum mean accuracy of 0.83 was obtained. When the automated results were compared with inter and intrarater accuracies, the automated results were shown to be comparable to the human raters suggesting that segmentation using DNNs is comparable to a second manual rater. As manually segmenting the retinal microvasculature is a tedious task, having a reliable automated output such as automated segmentation by DNNs, is an important step in creating an automated output.

In this paper a novel method has been proposed based on a combination of approximate computing, Discrete Wavelet Transform and deep neural network for image classification. In the recent trends, image classification using deep learning network comes under the limelight of the world of artificial intelligence. In the paper we have applied the approximation through bit width reduction technique through discrete wavelet transform technique for the processing of the input database. For feature extraction and classification we have developed a deep convolution neural network that is trained with that preprocesses data. The results show that the proposed model reduces the elapsed time and achieve a good rate of accuracy as compare to the Alexnet and Resnet-50 CNN models.

In recent years, the research of Multi-Agent Systems (MAS) has become a hot spot in complex systems research. It is widely used in multi-manipulator cooperative equipment, parallel computing, and network resource allocation. The planning of a multi-agent system is more and more critical. This paper takes RoboCup, a classic MAS, as an example. First, it is modeled mathematically through the Markov decision-making process, and a new ball-passing strategy solution based on a deep neural network is proposed. In addition, this paper also tries to use the traditional single-layer neural network to solve the same problem and compares the accuracy difference and model performance of the two types of neural networks. Finally, the possibility of applying a deep learning network in MAS is analyzed. The results show that the deep neural network has high effectiveness in solving the problem of the passing strategy of RoboCup. The ball control rate in the simulation environment is far more than the traditional algorithm, and the model trained by the deep learning network is significantly higher than the traditional neural network in accuracy, efficiency, and other aspects. Applying deep learning networks in other MAS is also likely to have a good effect. In the future, deep learning networks will be an essential planning algorithm.

The article considers the possibilities of using the deep learning convolutional neural network ResNet in computer vision and image classification problems. The interpretation of the ResNet network and the datasets used for its training are presented, as well as a method for training a deep convolutional neural network with stochastic depth, which allows significantly reducing errors in the test sample.

Recently, data-driven algorithms such as deep neural networks have attracted a lot of attention and have become a popular area of research and development. This interest is driven by several factors, such as recent advances in processing power (cheap and powerful hardware), the availability of large datasets (big data), and several small but important algorithmic advances (e.g., convolutional layers). Nowadays, deep neural networks are the state-of-the-art for several computer vision tasks, such as the ones that require high-level understanding of image semantics, e.g., image classification, object segmentation, saliency detection, and also in low level image processing tasks, such as image denoising, inpainting, and super-resolution, among others. Deep learning can handle large volumes of video data by making use of its powerful non-linear mapping and extracting high-level features with very deep networks. Incorporating deep learning into the IoVT can provide radical innovations in video sensing, coding, enhancing, understanding, and evaluation areas making it easier to handle the enormous growth in video data compared to traditional methods.

Automated image classification is an essential task of the computer vision field. The tagging of images into a set of predefined groups is referred to as image classification. The implementation of computer vision to automate image classification would be beneficial because manual image evaluation and identification can be time-consuming, particularly when there are many images of different classes. Deep learning approaches are proven to overperform existing machine learning techniques in many fields in recent years, and computer vision is one of the most notable examples. The very deep neural network (VDCNN) is a powerful deep learning model for image classification, and this paper examines it briefly using MNIST handwritten digit dataset. This dataset is used to prove the efficacy of very deep neural networks over other deep learning models. The proposed study aims to comprehend the very deep neural network architecture used to accomplish a handwritten digit recognition task. The feasibility of the proposed model is evaluated using mean accuracy, validation accuracy, and standard deviation. The study results of the very deep neural network model are compared to a convolutional neural network and convolutional neural network with batch normalization. According to the results of the comparison study, very deep neural networks achieve high accuracy of 99.1% for a handwritten dataset. The outcome of the proposed work is used to interpret how well a very deep neural network performs when compared to the other two models of deep neural networks. This proposed architecture may be used to automate the classification of handwritten digits dataset.

Inspired by the pioneering work of the information bottleneck (IB) principle for Deep Neural Networks’ (DNNs) analysis, we thoroughly study the relationship among the model accuracy, and , where and are the mutual information of DNN’s output T with input X and label Y. Then, we design an information plane-based framework to evaluate the capability of DNNs (including CNNs) for image classification. Instead of each hidden layer’s output, our framework focuses on the model output T. We successfully apply our framework to many application scenarios arising in deep learning and image classification problems, such as image classification with unbalanced data distribution, model selection, and transfer learning. The experimental results verify the effectiveness of the information plane-based framework: Our framework may facilitate a quick model selection and determine the number of samples needed for each class in the unbalanced classification problem. Furthermore, the framework explains the efficiency of transfer learning in the deep learning area.

Neural ordinary differential equations (neural ODEs) have emerged as a novel network architecture that bridges dynamical systems and deep learning. However, the gradient obtained with the continuous adjoint method in the vanilla neural ODE is not reverse-accurate. Other approaches suffer either from an excessive memory requirement due to deep computational graphs or from limited choices for the time integration scheme, hampering their application to large-scale complex dynamical systems. To achieve accurate gradients without compromising memory efficiency and flexibility, we present a new neural ODE framework, PNODE, based on high-level discrete adjoint algorithmic differentiation. By leveraging discrete adjoint time integrators and advanced checkpointing strategies tailored for these integrators, PNODE can provide a balance between memory and computational costs, while computing the gradients consistently and accurately. We provide an open-source implementation based on PyTorch and PETSc, one of the most commonly used portable, scalable scientific computing libraries. We demonstrate the performance through extensive numerical experiments on image classification and continuous normalizing flow problems. We show that PNODE achieves the highest memory efficiency when compared with other reverse-accurate methods. On the image classification problems, PNODE is up to two times faster than the vanilla neural ODE and up to 2.3 times faster than the best existing reverse-accurate method. We also show that PNODE enables the use of the implicit time integration methods that are needed for stiff dynamical systems.

We study the eigenvalue distribution of random matrices pertinent to the analysis of deep neural networks. The matrices resemble the product of the sample covariance matrices, however, an important difference is that the analog of the population covariance matrix is now a function of random data matrices (synaptic weight matrices in the deep neural network terminology). The problem has been treated in recent work [J. Pennington, S. Schoenholz and S. Ganguli, The emergence of spectral universality in deep networks, Proc. Mach. Learn. Res. 84 (2018) 1924–1932, arXiv:1802.09979] by using the techniques of free probability theory. Since, however, free probability theory deals with population covariance matrices which are independent of the data matrices, its applicability in this case has to be justified. The justification has been given in [L. Pastur, On random matrices arising in deep neural networks: Gaussian case, Pure Appl. Funct. Anal. (2020), in press, arXiv:2001.06188] for Gaussian data matrices with independent entries, a standard analytical model of free probability, by using a version of the techniques of random matrix theory. In this paper, we use another version of the techniques to extend the results of [L. Pastur, On random matrices arising in deep neural networks: Gaussian case, Pure Appl. Funct. Anal. (2020), in press, arXiv:2001.06188] to the case where the entries of the data matrices are just independent identically distributed random variables with zero mean and finite fourth moment. This, in particular, justifies the mean field approximation in the infinite width limit for the deep untrained neural networks and the property of the macroscopic universality of random matrix theory in this case.

RETRACTED: Deep learning network (DL-Net) based classification and segmentation of multi-class retinal diseases using OCT scans

Raman spectroscopy has attracted much attention due to its wide applications in drug detection and many other fields. However, Raman spectra often contain background noise, which poses significant challenges for subsequent analysis and processing. Although various methods for removing background noise have been proposed and have improved analysis accuracy to some extent, these mathematical model-based methods often rely on parameter adjustments based on spectral data to achieve the desired de-noising effect. The introduction of deep learning technology has provided new ideas to solve this problem, breaking through the limitations of traditional methods in parameter dependency. Different deep learning structures exhibit unique advantages when processing different types of data. In this study, we propose a static dropout triangular deep convolutional network (SD-TDCN). This deep learning network can maintain superior performance while significantly reducing the size of model parameters by statically discarding some convolutional blocks in the deep learning network. In addition, this deep learning network also lays an experimental foundation for the subsequent development of deep learning structures with adaptive dropout mechanisms.

The intelligent transportation system (ITS) has been proven capable of effectively addressing traffic congestion issues. For vehicles to perform effectively and improve mobility under the intelligent driving environment, real-time prediction of traffic speed is undoubtedly essential. Considering the complex spatiotemporal dependency inherent in traffic data, conventional prediction models encounter many limitations. To improve the prediction performance and investigate the temporal features, this study focuses on emerging deep neural networks (DNNs) using the Caltrans Performance Measurement System (PeMS) data. This research also establishes an intelligent driving environment in the simulation and compares the traditional car-following model with deep learning methods in terms of multiple performance metrics. The results indicate that both supervised learning and unsupervised learning are superior to the simulation-based model on the freeway, and the two deep learning networks are almost identical to one another. Besides, the result reveals that all models have their latent features for different time dimensions under the low traffic loads, transition states, and heavy traffic loads. This is critical in the application of prediction technologies in ITS. The findings can assist transportation researchers and traffic engineers in both traffic operation and management, such as bottleneck identification, platooning control, and route planning.