Baseline correction of Raman spectra using static dropout triangular deep convolutional network

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

Chronic liver disease (CLD) is currently one of the major causes of death worldwide. If not treated, it may lead to cirrhosis, hepatic carcinoma and death. Ultrasound (US) shear wave elastography (SWE) is a relatively new, popular, non-invasive technique among radiologists. Although many studies have been published validating the SWE technique either in a clinical setting, or by applying machine learning on SWE elastograms, minimal work has been done on comparing the performance of popular pre-trained deep learning networks on CLD assessment. Currently available literature reports suggest technical advancements on specific deep learning structures, with specific inputs and usually on a limited CLD fibrosis stage class group, with limited comparison on competitive deep learning schemes fed with different input types. The aim of the present study is to compare some popular deep learning pre-trained networks using temporally stable and full elastograms, with or without augmentation as well as propose suitable deep learning schemes for CLD diagnosis and progress assessment. 200 liver biopsy validated patients with CLD, underwent US SWE examination. Four images from the same liver area were saved to extract elastograms and processed to exclude areas that were temporally unstable. Then, full and temporally stable masked elastograms for each patient were separately fed into GoogLeNet, AlexNet, VGG16, ResNet50 and DenseNet201 with and without augmentation. The networks were tested for differentiation of CLD stages in seven classification schemes over 30 repetitions using liver biopsy as the reference. All networks achieved maximum mean accuracies ranging from 87.2%–97.4% and area under the receiver operating characteristic curves (AUCs) ranging from 0.979–0.990 while the radiologists had AUCs ranging from 0.800–0.870. ResNet50 and DenseNet201 had better average performance than the other networks. The use of the temporal stability mask led to improved performance on about 50% of inputs and network combinations while augmentation led to lower performance for all networks. These findings can provide potential networks with higher accuracy and better setting in the CLD diagnosis and progress assessment. A larger data set would help identify the best network and settings for CLD assessment in clinical practice.

Detecting subtle signals from small earthquakes triggered by transient stresses from the surface waves of large magnitude earthquakes can contribute to a more general understanding of how earthquakes nucleate and interact with each other. However, searching for signals from such small earthquakes in thousands of seismograms is overwhelming, and discriminating them from a miscellany of noise is challenging. Here, we explore how we can automate the detection of such dynamically triggered earthquakes using a simple, diagnostic signal-to-noise ratio (SNR) threshold as well as a convolutional deep metric learning network. Our analysis shows that the deep learning network was more reliable at detecting small earthquakes than the SNR method.

Deep learning neural network approach to thermal-wave imaging of damage in solids with application to diffusivity measurements of a green (unsintered) metal powder compact slab

Deep learning is a recently developed feature representation technique for data with complicated structures, which has great potential for soft sensing of industrial processes. However, most deep networks mainly focus on hierarchical feature learning for the raw observed input data. For soft sensor applications, it is important to reduce irrelevant information and extract quality-relevant features from the raw input data for quality prediction. To deal with this problem, a novel deep learning network is proposed for quality-relevant feature representation in this article, which is based on stacked quality-driven autoencoder (SQAE). First, a quality-driven autoencoder (QAE) is designed by exploiting the quality data to guide feature extraction with the constraint that the potential features should largely reconstruct the input layer data and the quality data at the output layer. In this way, quality-relevant features can be captured by QAE. Then, by stacking multiple QAEs to construct the deep SQAE network, SQAE can gradually reduce irrelevant features and learn hierarchical quality-relevant features. Finally, the high-level quality-relevant features can be directly applied for soft sensing of the quality variables. The effectiveness and flexibility of the proposed deep learning model are validated on an industrial debutanizer column process.

BackgroundTo determine if mammographic features from deep learning networks can be applied in breast cancer to identify groups at interval invasive cancer risk due to masking beyond using traditional breast density measures.MethodsFull-field digital screening mammograms acquired in our clinics between 2006 and 2015 were reviewed. Transfer learning of a deep learning network with weights initialized from ImageNet was performed to classify mammograms that were followed by an invasive interval or screen-detected cancer within 12 months of the mammogram. Hyperparameter optimization was performed and the network was visualized through saliency maps. Prediction loss and accuracy were calculated using this deep learning network. Receiver operating characteristic (ROC) curves and area under the curve (AUC) values were generated with the outcome of interval cancer using the deep learning network and compared to predictions from conditional logistic regression with errors quantified through contingency tables.ResultsPre-cancer mammograms of 182 interval and 173 screen-detected cancers were split into training/test cases at an 80/20 ratio. Using Breast Imaging-Reporting and Data System (BI-RADS) density alone, the ability to correctly classify interval cancers was moderate (AUC = 0.65). The optimized deep learning model achieved an AUC of 0.82. Contingency table analysis showed the network was correctly classifying 75.2% of the mammograms and that incorrect classifications were slightly more common for the interval cancer mammograms. Saliency maps of each cancer case found that local information could highly drive classification of cases more than global image information.ConclusionsPre-cancerous mammograms contain imaging information beyond breast density that can be identified with deep learning networks to predict the probability of breast cancer detection.

Deep learning networks were employed to predict the maximum differential deflection of stiffened and waffle rafts due to reactive soil movements, Δmax. Four deep learning networks were used to predict Δmax, these are (1) stiffened rafts on shrinking soil, (2) stiffened rafts on swelling soil, (3) waffle rafts on shrinking soil, and (4) waffle rafts on swelling soil. The deep learning models were used to create design lines, which showed that both soil and structural features strongly influence the stiffened rafts. In contrast, waffle rafts showed a strong dependence on soil features in shrinking soils and beam depth in swelling soils. This demonstrates that the finite element-based deep learning networks captured the effect of the embedment of the beams. The results of the deep learning models led to non-linear design curves, which are disparate from the suggested standard Australian design. These results suggest that increasing the value of beam depth can have a positive or negative impact on the global residential slab depending on the type of substructure and whether the founding reactive soil is shrinking or swelling. Global sensitivity analyses of the deep learning models showed that for stiffened rafts on shrinking soil, the slab length, slab width and active depth zone of reactive soil had the most significant influence on Δmax, whilst for stiffened rafts on swelling soil, the primary drivers are ground movement, beam depth, and slab width. The prediction of Δmax for waffle rafts on shrinking soil was driven by the surface characteristic and mound movements, and the active depth zone, whilst waffle rafts on swelling soil was driven by the beam depth. Overall, the finite element-based deep learning showed the capacity to estimate Δmax in both shrinking and swelling design scenarios for different types of residential footing systems to further understand the characteristic behaviour of shallow residential slab foundations on reactive soils leading to improved designs.

Raman spectroscopy is an efficient method for detection of explosives even in small quantities. A laser can be combined with a Coded Aperture Snapshot Spectral Imaging (CASSI) system to collect Raman spectra from a surface at stand-off distances. The CASSI-system decrease the data collection time but instead increase the reconstruction time for the Raman image. Reconstruction of Raman spectra from an ensemble of compressed sensing measurements using standard reconstruction methods such as Total Variation (TV) is rather time consuming and limits the application domain for the technique. Novel machine learning approaches such as Deep Learning (DL) has lately been applied to reconstruction problems. We evaluate our earlier developed DL approach for reconstruction of Raman spectra from an ensemble of measurements formulated as a regression problem. The DL network is trained by minimizing a loss-function which is composed of two components: a reconstruction error and a re-projection error. The evaluated method is trained on simulated data where the training data has been generated using a transfer function. The transfer function has been developed to mimic the optical properties of a CASSI system. The DL network has been trained on different training sets with different levels of background noise, different number of materials in the scene and different spatial configurations of the materials in the scene. The reconstruction results using the DL network has been qualitatively evaluated on simulated data and the results are also compared to the Two-Step Iterative Shrinkage/Thresholding (TwIST) algorithm in terms of reconstruction quality and computation time. The reconstruction time for the DL are orders of magnitude lower than for TwIST without reducing the quality of the reconstructed Raman spectra.

Big data is playing a significant role in the current computing revolution. Industries and organizations are utilizing their insights for Business Intelligence by using Deep Learning Networks (DLN). However, dynamic characteristics of BD introduce many critical issues for DLN; Concept Drift (CD) is one of them. CD issue appears frequently in Online Supervised Learning environments in which data trends change over time. The problem may even worsen in a BD environment due to the veracity and variability factors. The CD issue may render the DLN inapplicable by degrading the accuracy of classification results in DLN which is a very serious issue that needs to be addressed. Therefore, these DLN need to quickly adapt to changes for maintaining the accuracy level of the results. To overcome classification accuracy, we need some dynamical changes in the existing DLN. Therefore, in this paper, we examine some of the existing Shallow Learning and Deep Learning models and their behavior before and after the Concept Drift (in experiment 1) and validate the pre-trained Deep Learning network (ResNet-50). In future work, this experiment will examine the most recent pre-trained DLN (Alex Net, VGG16, VGG19) and identify their suitability to overcome Concept Drift using fine-tuning and transfer learning approaches.

Mutual-guided scale-aggregation denoising network for seismic noise attenuation

Knowledge-driven building extraction method exhibits a restricted adaptability scope and is vulnerable to external factors that affect its extraction accuracy. On the other hand, data-driven building extraction method lacks interpretability, heavily relies on extensive training data, and may result in extraction outcomes with building boundary blur issues. The integration of pre-existing knowledge with data-driven learning is essential for the intelligent identification and extraction of buildings from high-resolution aerial images. To overcome the limitations of current deep learning building extraction networks in effectively leveraging prior knowledge of aerial images, a geometric significance-aware deep mutual learning network (GSDMLNet) is proposed. Firstly, the GeoSay algorithm is utilized to derive building geometric significance feature maps as prior knowledge and integrate them into the deep learning network to enhance the targeted extraction of building features. Secondly, a bi-directional guidance attention module (BGAM) is developed to facilitate deep mutual learning between the building feature map and the building geometric significance feature map within the dual-branch network. Furthermore, the deployment of an enhanced flow alignment module (FAM++) is utilized to produce high-resolution, robust semantic feature maps with strong interpretability. Ultimately, a multi-objective loss function is crafted to refine the network’s performance. Experimental results demonstrate that the GSDMLNet excels in building extraction tasks within densely populated and diverse urban areas, reducing misidentification of shadow-obscured regions and color-similar terrains lacking building structural features. This approach effectively ensures the precise acquisition of urban building information in aerial images.

Deep learning algorithm has been widely used in many area which is one of the most important representation learning algorithms in machine learning tasks. Deep learning network is stacked by the building blocks such as the restricted Boltzmann machine(RBM) and the auto-encoder, convolutional building block. After stacking the building blocks layers and layers, the improvement of the deep learning network would be notable. In this paper, we proposed a new deep learning building block that inspired by the auto-encoder, which is the compressed auto-encoder with fewer layers and parameters compared with the auto-encoder, and we put forward a bidirectional gradient decent method to update the parameters of this building block. As the experimental results show that improves the performance of the auto-encoder in accuracy of the reconstruction data. It keeps declining the error while the results of rbm or the auto-encoder becomes saturation, and some analysis are given in this paper.

(1) Background: The effectiveness of deep learning artificial intelligence depends on data availability, often requiring large volumes of data to effectively train an algorithm. However, few studies have explored the minimum number of images needed for optimal algorithmic performance. (2) Methods: This institutional review board (IRB)-approved retrospective review included patients who received prostate magnetic resonance imaging (MRI) between September 2014 and August 2018 and a magnetic resonance imaging (MRI) fusion transrectal biopsy. T2-weighted images were manually segmented by a board-certified abdominal radiologist. Segmented images were trained on a deep learning network with the following case numbers: 8, 16, 24, 32, 40, 80, 120, 160, 200, 240, 280, and 320. (3) Results: Our deep learning network’s performance was assessed with a Dice score, which measures overlap between the radiologist’s segmentations and deep learning-generated segmentations and ranges from 0 (no overlap) to 1 (perfect overlap). Our algorithm’s Dice score started at 0.424 with 8 cases and improved to 0.858 with 160 cases. After 160 cases, the Dice increased to 0.867 with 320 cases. (4) Conclusions: Our deep learning network for prostate segmentation produced the highest overall Dice score with 320 training cases. Performance improved notably from training sizes of 8 to 120, then plateaued with minimal improvement at training case size above 160. Other studies utilizing comparable network architectures may have similar plateaus, suggesting suitable results may be obtainable with small datasets.

Background: Over the last several decades, predicting protein structures from amino acid sequences has been a core task in bioinformatics. Nowadays, the most successful methods employ multiple sequence alignments and can predict the structure with excellent performance. These predictions take advantage of all the amino acids at a given position and their frequencies. However, the effect of single amino acid substitutions in a specific protein tends to be hidden by the alignment profile. For this reason, single-sequence-based predictions attract interest even after accurate multiple-alignment methods have become available: the use of single sequences ensures that the effects of substitution are not confounded by homologous sequences. Objective: This work aims at understanding how the single-sequence secondary structure prediction of a residue is influenced by the surrounding ones. We aim at understanding how different prediction methods use single-sequence information to predict the structure. Methods: We compare mutual information, the coefficients of two linear models, and three deep learning networks. For the deep learning algorithms, we use the DeepLIFT analysis to assess the effect of each residue at each position in the prediction. Result: Mutual information and linear models quantify direct effects, whereas DeepLIFT applied on deep learning networks quantifies both direct and indirect effects. Conclusion: Our analysis shows how different network architectures use the information of single protein sequences and highlights their differences with respect to linear models. In particular, the deep learning implementations take into account context and single position information differently, with the best results obtained using the BERT architecture.

Technical note: Impact of tissue section thickness on accuracy of cell classification with a deep learning network.