Machine learning approaches for crystallographic classification from synthetic 2D X-ray diffraction data

Here we report a facile, prompt protocol based on deep-learning techniques to sort out intricate phase identification and quantification problems in complex multiphase inorganic compounds. We simulate plausible powder X-ray diffraction (XRD) patterns for 170 inorganic compounds in the Sr-Li-Al-O quaternary compositional pool, wherein promising LED phosphors have been recently discovered. Finally, 1,785,405 synthetic XRD patterns are prepared by combinatorically mixing the simulated powder XRD patterns of 170 inorganic compounds. Convolutional neural network (CNN) models are built and eventually trained using this large prepared dataset. The fully trained CNN model promptly and accurately identifies the constituent phases in complex multiphase inorganic compounds. Although the CNN is trained using the simulated XRD data, a test with real experimental XRD data returns an accuracy of nearly 100% for phase identification and 86% for three-step-phase-fraction quantification.

This study presents a novel approach using Wasserstein Generative Adversarial Networks with Gradient Penalty (WGAN-GP) to generate synthetic electroencephalography (EEG) and electrocardiogram (ECG) waveforms. The synthetic EEG data represent concentration and relaxation mental states, while the synthetic ECG data correspond to normal and abnormal states. By addressing the challenges of limited biophysical data, including privacy concerns and restricted volunteer availability, our model generates realistic synthetic waveforms learned from real data. Combining real and synthetic datasets improved classification accuracy from 92% to 98.45%, highlighting the benefits of dataset augmentation for machine learning performance. The WGAN-GP model achieved 96.84% classification accuracy for synthetic EEG data representing relaxation states and optimal accuracy for concentration states when classified using a fusion of convolutional neural networks (CNNs). A 50% combination of synthetic and real EEG data yielded the highest accuracy of 98.48%. For EEG signals, the real dataset consisted of 60-s recordings across four channels (TP9, AF7, AF8, and TP10) from four individuals, providing approximately 15,000 data points per subject per state. For ECG signals, the dataset contained 1200 real samples, each comprising 140 data points, representing normal and abnormal states. WGAN-GP outperformed a basic generative adversarial network (GAN) in generating reliable synthetic data. For ECG data, a support vector machine (SVM) classifier achieved an accuracy of 98% with real data and 95.8% with synthetic data. Synthetic ECG data improved the random forest (RF) classifier’s accuracy from 97% with real data alone to 98.40% when combined with synthetic data. Statistical significance was assessed using the Wilcoxon signed-rank test, demonstrating the robustness of the WGAN-GP model. Techniques such as discrete wavelet transform, downsampling, and upsampling were employed to enhance data quality. This method shows significant potential in addressing biophysical data scarcity and advancing applications in assistive technologies, human-robot interaction, and mental health monitoring, among other medical applications.

Deep learning (DL) applications of seismic reservoir characterization often require the generation of synthetic data to augment available sparse labeled data. An approach for generating synthetic training data consists of specifying probability distributions modeling prior geologic uncertainty on reservoir properties and forward modeling the seismic data. A prior falsification approach is critical to establish the consistency of the synthetic training data distribution with real seismic data. With the help of a real case study of facies classification with convolutional neural networks (CNNs) from an offshore deltaic reservoir, we have highlighted several practical nuances associated with training DL models on synthetic seismic data. We highlight the issue of overfitting of CNNs to the synthetic training data distribution and propose regularization strategies to address it. We demonstrate the efficacy of our proposed strategies by training the CNN on synthetic data and making robust predictions with real 3D partial stack seismic data.

This study assesses the effectiveness of deep learning models, specifically Recurrent Neural Network (RNN), Convolutional Neural Network (CNN), and Long Short-Term Memory (LSTM), in assessing synthetic grip strength data for stress ball and pencil grips. The aim is to determine the optimal model for grip strength classification and tackle issues such as temporal data and class imbalance. Synthetic data was created, preprocessed, and utilized to train the models on Google Colab employing TensorFlow and Keras. Hyperparameters, such as CNN kernel dimensions and RNN/LSTM sequence lengths, were refined. Performance was evaluated by accuracy, precision, F1 scores, and loss metrics. The RNN model surpassed both CNN and LSTM, with an accuracy of 94.99% and a precision of 0.56, particularly excelling in the classification of “Normal” grip strength instances. CNN and LSTM had difficulties with time dependencies and class imbalance, leading to reduced accuracy and increased loss. Although RNN has proven to be the most effective approach, subsequent research should concentrate on mitigating class imbalance and investigating hybrid models to improve performance.

Marine seismic data is often missing near offset information due to separation between the source and receiver cables. To solve this problem, a convolutional neural network is trained on synthetic seismic data to reconstruct the near offset gap. The synthetic data is created using a two‐dimensional finite difference method within a heterogeneous velocity model. These synthetics are generated with a source‐over‐receiver acquisition geometry so that they contain complete near offset data. The convolutional neural network is then trained on input‐target synthetic pairs where the inputs are common midpoint gathers with the near offset section removed, and the targets are the same gathers with the near offset section retained. Following training, the robustness of the method is investigated with regards to common midpoint data sorting, normal moveout correction and changes in the velocity model. It is found that training on common midpoint‐sorted data results in 2.8 times lower error than training on shot gathers, that normal moveout correction of the training data makes no significant difference in error levels, and that the model can reconstruct realistic near offsets on synthetic data generated 10 km away within the heterogeneous velocity model. In field data testing, first a dataset with source‐over‐cable acquisition geometry from the Barents Sea is used to compare the reconstructed wavefields to ground truth values. Although the reconstructed amplitudes require minor scaling to match the true values, predictions on this dataset yield 2.5 times lower near offset reconstruction error compared to a simple Radon transform interpolation method. Furthermore, amplitude versus offset gradient and intercept sections from the Barents Sea dataset are estimated with half the error when including the convolutional neural network‐predicted near offset data, compared to only using the conventionally‐acquirable portion of the data (beyond 112.5 m of offset). In a secondary field data test, a conventional northern North Sea dataset is used to demonstrate how the method may be applied in practice. Here, the convolutional neural network generates more realistic predictions than the Radon method, and the gradient and intercept sections calculated using the convolutional neural network‐predicted traces have higher signal‐to‐noise ratios than the sections calculated using only the original data. The combination of high‐quality synthetic training data and interpolation in the common midpoint domain enables near offset reconstruction at significant depth (1 s of two‐way traveltime or more), which is demonstrated in both synthetic and field examples.

IntroductionPhotogrammetric surface scans provide a radiation-free option to assess and classify craniosynostosis. Due to the low prevalence of craniosynostosis and high patient restrictions, clinical data are rare. Synthetic data could support or even replace clinical data for the classification of craniosynostosis, but this has never been studied systematically.MethodsWe tested the combinations of three different synthetic data sources: a statistical shape model (SSM), a generative adversarial network (GAN), and image-based principal component analysis for a convolutional neural network (CNN)–based classification of craniosynostosis. The CNN is trained only on synthetic data but is validated and tested on clinical data.ResultsThe combination of an SSM and a GAN achieved an accuracy of 0.960 and an F1 score of 0.928 on the unseen test set. The difference to training on clinical data was smaller than 0.01. Including a second image modality improved classification performance for all data sources.ConclusionsWithout a single clinical training sample, a CNN was able to classify head deformities with similar accuracy as if it was trained on clinical data. Using multiple data sources was key for a good classification based on synthetic data alone. Synthetic data might play an important future role in the assessment of craniosynostosis.

In recent years, Convolutional Neural Networks (CNNs) have been widely successful for numerous computer vision tasks using 2D color and 3D point data, such as object detection and classification, and 3D pose estimation. Despite these successes, it comes with a price --- CNNs require a vast amount of labeled training data to be successful. Many training datasets are now available, however, they are mostly limited to general objects in the public domain such as people, cars, buildings, fruit, and more. Thus, not all fields have sufficient training data. Even if an adequate number of data samples can be built into a dataset, manually collecting and labeling the training data remains a laborious task. Several approaches to reduce or eradicate the need for training data have been proposed by researchers. The two most popular approaches are training with synthetic data and training on generalized object features. Training with synthetic data refers to the use of computer graphics to generate training data. While this approach has demonstrated success, it comes with its own challenges. In contrast, training a CNN on generalized object features identifies specific local 3D features or control points on objects. Local 3D features and their descriptors have proven in the past to be universally applicable and effective for general object detection, and thus, this method allows for a generalized CNN. A generalized CNN refers to a network architecture that, after initial training, can be specialized for a new task with a minimum amount or no new training data. This dissertation investigates an encoder-decoder architecture and determines the capabilities of it to generalize the CNN architecture for object detection tasks in point cloud data. A CNN is trained to recognize 3D features and demonstrates, without retraining, its successful applicability for point cloud object detection using disparate data. Additionally, the geometric consistency of the descriptors is evaluated to obtain further insight. The results demonstrate that the proposed 3D feature descriptors, and their increased geometric consistency contribute to the increased encoder-decoder architecture performance. In summary, this dissertation contributes a state-of-the-art, generalized, feature descriptor-based CNN architecture that can be transferred to different objects without retraining. Furthermore, it provides insight explaining the increased performance of the CNN compared to the state-of-the-art.

Chemical heuristics have been fundamental to the advancement of chemistry and materials science. These heuristics are typically established by scientists using knowledge and creativity to extract patterns from limited datasets. Machine learning offers opportunities to perfect this approach using computers and larger datasets. Here, we discuss the relationships between traditional heuristics and machine learning approaches. We show how traditional rules can be challenged by large-scale statistical assessment and how traditional concepts commonly used as features are feeding the machine learning techniques. We stress the waste involved in relearning chemical rules and the challenges in terms of data size requirements for purely data-driven approaches. Our view is that heuristic and machine learning approaches are at their best when they work together.

Reservoir characterization through seismic data analysis is essential for exploration and production in the petroleum industry. However, seismic-to-well tie discrepancies, limited availability of high-quality well data, and resolution constraints pose a reliability challenge. While previous studies offer valuable insights, they still struggle to achieve high-resolution predictions in a complex geologically environment given high reliance on well data. This study integrates synthetic data-driven techniques with real data, including convolutional neural networks (CNN) and transfer learning, to improve seismic reservoir characterization. We utilize nearby well statistics and a rock physics model (RPM) to simulate pseudo wells representing various geological scenarios. Synthetic seismic gathers are generated from these pseudo wells, which are based on RPM and local well control, to train the CNN. Transfer learning is then applied to adapt the CNN to better distinguish between real and synthetic data, enhancing reservoir predictions. A comparative analysis of P-impedance predictions from three methodologies: theory-driven Pre-Stack-Seismic-Inversion (TDSI), Deep-Neural-Network (DNN), and our CNN approach, showed that CNN achieved nearly 97% prediction accuracy with low error rates, compared to relatively lower prediction accuracy rates of DNN (86.2%) and TDSI (81.5%) with high error rates, according to robust metrics including R-square, RMSE, MSE, and MAE. These results indicate that CNN not only enhanced resolution but also closely aligned with well data and superior lateral continuity, even in blind well scenarios. This study effectively integrates synthetic data-driven techniques with CNNs and transfer learning to advance seismic reservoir property prediction, offering a robust solution to overcome limitations in traditional and DNN-based approaches.

Convolutional neural network and level-set spectral element method for ultrasonic imaging of delamination cavities in an anisotropic composite structure

FCSNet and ImFCSNet, convolutional neural network approaches to FCS data analysis.

Computer algorithms that simulate lower-doses computed tomography (CT) images from clinical-dose images are widely available. However, most operate in the projection domain and assume access to the reconstruction method. Access to commercial reconstruction methods may often not be available in medical research, making image-domain noise simulation methods useful. However, the introduction of non-linear reconstruction methods, such as iterative and deep learning-based reconstruction, makes noise insertion in the image domain intractable, as it is not possible to determine the noise textures analytically. To develop a deep learning-based image-domain method to generate low-dose CT images from clinical-dose CT (CDCT) images for non-linear reconstruction methods. We propose a fully image domain-based method, utilizing a series of three convolutional neural networks (CNNs), which, respectively, denoise CDCT images, predict the standard deviation map of the low-dose image, and generate the noise power spectra (NPS) of local patches throughout the low-dose image. All three models have U-net-based architectures and are partly or fully three-dimensional. As a use case for this study and with no loss of generality, we use paired low-dose and clinical-dose brain CT scans. A dataset of paired scans was retrospectively obtained. All images were acquired with a wide-area detector clinical system and reconstructed using its standard clinical iterative algorithm. Each pair was registered using rigid registration to correct for motion between acquisitions. The data was randomly partitioned into training ( samples), validation ( samples), and test ( samples) sets. The performance of each of these three CNNs was validated separately. For the denoising CNN, the local standard deviation decrease, and bias were determined. For the standard deviation map CNN, the real and estimated standard deviations were compared locally. Finally, for the NPS CNN, the NPS of the synthetic and real low-dose noise were compared inside and outside the skull. Two proof-of-concept denoising studies were performed to determine if the performance of a CNN- or a gradient-based denoising filter on the synthetic low-dose data versus real data differed. The denoising network had a median decrease in noise in the cerebrospinal fluid by a factor of and introduced a median bias of HU. The network for standard deviation map estimation had a median error of HU. The noise power spectrum estimation network was able to capture the anisotropic and shift-variant nature of the noise structure by showing good agreement between the synthetic and real low-dose noise and their corresponding power spectra. The two proof of concept denoising studies showed only minimal difference in standard deviation improvement ratio between the synthetic and real low-dose CT images with the median difference between the two being 0.0 and +0.05 for the CNN- and gradient-based filter, respectively. The proposed method demonstrated good performance in generating synthetic low-dose brain CT scans without access to the projection data or to the reconstruction method. This method can generate multiple low-dose image realizations from one clinical-dose image, so it is useful for validation, optimization, and repeatability studies of image-processing algorithms.

Automatic waveform classification and arrival picking methods are widely studied to reduce or replace the manual works. Machine learning based methods, especially neural networks, and clustering based methods have shown great potentials in previous studies. However, most of the existing methods are sensitive to noise. The convolution neural networks (CNNs), developed from the traditional neural networks, have been successfully applied in many different fields, but are rarely studied in seismic waveform classification. In this paper, we propose a novel antinoise CNN architecture for waveform classification and also propose to combine k‐means clustering (KC) with CNN classification to pick arrivals (CNN‐KC). Seismic data are sampled to 1‐D vectors using a specific time window. Using the trained CNN classifier, these 1‐D vectors are classified into two categories: waveform and nonwaveform. With the constraint of the first waveform, CNN‐KC can pick the arrival more accurately. We also apply the proposed methods to the synthetic microseismic data with different noise levels and the actual field microseismic data to test their robustness. CNNs perform much better than the traditional multilayer perceptron on the waveform classification of the noisy microseismic data. Based on the analysis of the CNN internal architecture, we also conclude that the main reason that CNN is insensitive to noise is the convolution and pooling layers which behave like smooth operation in some ways. The final results show that the CNN and CNN‐KC are effective and robust methods for waveform classification and arrival picking.

The ability of Convolutional Neural Networks (CNNs) to learn from vast amounts of data and improve accuracy over time makes them an attractive solution for many industrial problems. In the context of Future Assembly Systems such as Line-Less Mobile Assembly Systems, CNNs can be used to monitor the networked system of mobile robots, human operators, and other movable objects that assemble products in flexible environment configurations. This paper explores the use of a simulated industrial environment to autonomously generate training data for object detection, tracking, and segmentation CNNs. The goal is to adapt state-of-the-art CNN solutions to specific industry use cases, where real data annotation can be time-consuming and expensive. The developed algorithm efficiently generates new random image data, allowing accurate object detection, tracking, and segmentation in dynamic industrial scenarios. The results show the effectiveness of this approach in improving the testing of CNNs for industrial applications.

Polar transformation of 2D X-ray diffraction patterns and the experimental validation of the hDIC technique