Large-Scale Industrial Deployment of Machine Learning Workflows for Seismic Data Processing

Seismic data reconstruction plays an important role in the whole seismic data processing and imaging workflow, especially for those data that are acquired from severe field environment and are missing a large portion of the reflection signals. The rank-reduction method is considered to be a very effective method for interpolating data that are of small curvature, e.g. the post-stack data. However, when the data are more complicated, the rank-reduction method may fail to achieve acceptable performance. A useful strategy is to use local windows to process the data so that the data in each local window satisfy the plane-wave assumption of the rank-reduction method. However, the rank in each window requires a careful selection. Traditional methods select a global rank for all windows. We have proposed an automatic algorithm to select the rank in each processing window. The energy ratio between two consecutive singular values is chosen as the criterion to define the optimal rank. We apply this strategy to seismic data interpolation and use both synthetic and field data examples to demonstrate its potential in practical applications.

Iterative deblending of simultaneous-source data using smoothed singular spectrum analysis

The paper is focused on research in the area of building large datasets using Apache Hadoop. Our team is managing an information system that is able to calculate probability of existence of different objects in space and time. The system works with a lot of different data sources, including large datasets. The workflow of data processing is quite complicated and time consuming, so we were looking for some framework that could help with system management and, if possible, to speed up data processing as well. Apache Hadoop was selected as a platform for enhance our information system. Apache Hadoop is usually used for processing large datasets, but in a case of our information system is necessary to perform other types of tasks as well. The systems computes spatio-temporal relations between different types of objects. This means that from relatively small amount of records (thousands) are built relatively large datasets (millions of records). For this purposes is usually used PostgreSQL/PostGIS database or tools written in Java or other language. Our research was focused to determination if we could simply move some of this tasks to Apache Hadoop platform using simple SQL editor like Hive. We have selected two types of common tasks and tested them on PostgreSQL and Apache Hadoop (Hive) platform to be able compare time necessary to complete these tasks. The paper presents results of our research.

Digital oil and gas field is an overly complex integrated information system, and with the continuous expansion of business scale and needs, oil companies will constantly raise more new and higher requirements for digital transformation. In the previous system construction, we adopted multi-phase, multi-vendor, multi-technology and multi-method, resulting in the problem of data silos and fragmentation. The result of the data management problems is that decisions are often made using incomplete information. Even when the desired data is accessible, requirements for gathering and formatting it may limit the amount of analysis performed before a timely decision must be made. Therefore, through the use of advanced computer technologies such as big data, cloud computing and IOT (internet of things), it has become our current goal to build an integrated data integration platform and provide unified data services to improve the company's bottom line. As part of the digital oilfield, offshore drilling operations is one of the potential areas where data processing and advanced analytics technology can be used to increase revenue, lower costs, and reduce risks. Building a data mining and analytics engine that uses multiple drilling data is a difficult challenge. The workflow of data processing and the timeliness of the analysis are major considerations for developing a data service solution. Most of the current analytical engines require more than one tool to have a complete system. Therefore, adopting an integrated system that combines all required tools will significantly help an organization to address the above challenges in a timely manner. This paper serves to provide a technical overview of the offshore drilling data service system currently developed and deployed. The data service system consists of four subsystems. They are the static data management system including structured data (job report) and unstructured data (design documentation and research report), the real-time data management system, the third-party software data management system integrating major industry software databases, and the cloud-based data visual application system providing dynamic analysis results to achieve timely optimization of the operations. Through a unified logical data model, it can realize the quick access to the third-party software data and application support; These subsystems are fully integrated and interact with each other to function as microservices, providing a one-stop solution for real-time drilling optimization and monitoring. This data service system has become a powerful decision support tool for the drilling operations team. The learned lessons and gained experiences from the system services presented here provide valuable guidance for future demands E&P and the industrial revolution.

Rapidly growing data volume demand systems that have the ability to handle massive heterogeneous unstructured data sets in digital home environment. However, currently, most existing mature transaction processing systems are built upon relational databases with structured data. In this paper, we design a hybrid development framework, to offer greater scalability and flexibility of data analysis and report, while keeping maximum compatibilities and links to the legacy platforms on which transaction business logics run. Data, service and user interfaces are implemented as a toolset stack, for developing applications with functionalities of information retrieval, data processing, analysing and visualizing. We discuss a use case of healthcare data integration as an example, where information is collected and aggregated from diverse sources such as electronic health records (EHRs), personal health records (PHRs) and wearable devices. We present the workflow of data processing and visualization, to validate the effectiveness of the proposed framework.

Time‐lapse seismic data processing is an important technique for observing subsurface changes over time. The conventional time‐lapse seismic exploration has been conducted using a large‐scale exploration system. However, for efficient monitoring of shallow subsurface, time‐lapse monitoring based on the small‐scale exploration system is required. Small‐scale exploration system using a sparker source offers high vertical resolution and cost efficiency, but it faces challenges, such as inconsistent waveforms of sparker sources, inaccurate positioning information and a low signal‐to‐noise ratio. Therefore, this study proposes a data processing workflow to preserve the signal and enhance the repeatability of small‐scale time‐lapse seismic data acquired using a sparker source. The proposed workflow has three stages: pre‐stack, post‐stack and machine learning–based data processing. Conventional seismic data processing methods were applied to enhance the quality of the sparker seismic data during the pre‐stack data processing stage. In the post‐stack processing stage, the positions and energy correction were performed, and the machine learning–based data processing stage attenuated random noise and applied a matched filter. The data processing was performed using only the seismic signals recorded near the seafloor, and the results confirmed the improvement in the repeatability of the entire seismic profile, including that of the target area. According to the repeatability quantification results, the predictability increased and the normalized root mean square decreased during data processing, indicating improved repeatability. In particular, the repeatability of the data was greatly improved through vertical correction, energy correction and matched filtering approaches. The processing results demonstrate that the data processing method proposed in this study can effectively enhance the repeatability of high‐resolution time‐lapse seismic data. Consequently, this approach could contribute to a more accurate understanding of temporal changes in subsurface structure and material properties.

Technology drives advances in science. Giving scientists access to more powerful tools for collecting and understanding data enables them to both ask and answer new kinds questions that were previously beyond their reach. Of these new tools at their disposal, machine learning offers the opportunity to understand and analyze data at unprecedented scales and levels of detail. The standard machine learning pipeline consists of data labeling, feature extraction, training, and evaluation. However, without expert machine learning knowledge, it is difficult for scientists to optimally construct this pipeline to fully leverage machine learning in their work. Using ecology as a motivating example, we analyze a typical scientist's data collection and processing workflow and highlight many problems facing practitioners when attempting to capitalize on advances in machine learning and pattern recognition. Understanding these shortcomings allows us to outline several novel and underexplored research directions. We end with recommendations to motivate progress in future cross-disciplinary work.

The offshore survey industry continues to develop and introduce new technologies. Significant advancements in a variety of technologies have led to the successful introduction of AUV's for a variety of surveying roles. However their impact on the “high end” pipeline inspection is not yet complete. Can AUV's become the standard acquisition platform for pipeline and other inspection surveys or are the challenges and obstacles that prevent the uptake and adoption of AUV technology simply too great? There are now component technologies that, when properly constructed, enable the use of AUV technology for many more survey and inspection related activities. This includes the collection, processing and management of the survey sensor data sets. AUV's are now capable of carrying such sensors as the latest high resolution multibeam sonars, synthetic aperture sonars, high definition cameras and acoustic doppler current profilers. With this array of available acoustic sensors we can expect AUV's to be used for an increasing number of pipeline inspection surveys and other hydrographic survey missions. This increased usage will likely require new data processing workflows and techniques, especially in consideration of the huge data volumes that will require processing once the vehicle returns to its parent ship and data is downloaded. This paper will explore these processing workflows and highlight the challenges and benefits with a view to addressing some key questions and promoting discussion on the future use of AUVs.

Material characterization techniques are widely used to characterize the physical and chemical properties of materials at the nanoscale and, thus, play central roles in material scientific discoveries. However, the large and complex datasets generated by these techniques often require significant human effort to interpret and extract meaningful physicochemical insights. Artificial intelligence (AI) techniques such as machine learning (ML) have the potential to improve the efficiency and accuracy of surface analysis by automating data analysis and interpretation. In this perspective paper, we review the current role of AI in surface analysis and discuss its future potential to accelerate discoveries in surface science, materials science, and interface science. We highlight several applications where AI has already been used to analyze surface analysis data, including the identification of crystal structures from XRD data, analysis of XPS spectra for surface composition, and the interpretation of TEM and SEM images for particle morphology and size. We also discuss the challenges and opportunities associated with the integration of AI into surface analysis workflows. These include the need for large and diverse datasets for training ML models, the importance of feature selection and representation, and the potential for ML to enable new insights and discoveries by identifying patterns and relationships in complex datasets. Most importantly, AI analyzed data must not just find the best mathematical description of the data, but it must find the most physical and chemically meaningful results. In addition, the need for reproducibility in scientific research has become increasingly important in recent years. The advancement of AI, including both conventional and the increasing popular deep learning, is showing promise in addressing those challenges by enabling the execution and verification of scientific progress. By training models on large experimental datasets and providing automated analysis and data interpretation, AI can help to ensure that scientific results are reproducible and reliable. Although integration of knowledge and AI models must be considered for the transparency and interpretability of models, the incorporation of AI into the data collection and processing workflow will significantly enhance the efficiency and accuracy of various surface analysis techniques and deepen our understanding at an accelerated pace.

While much of the workflow for geophysical data processing and analysis is automated, final quality assurance typically requires the decisions of skilled human analysts and interpreters. The quality and reliability of geophysical inverse models depends directly on the effectiveness of input data. The influence of spurious data on derived data products used in the inversion has been reduced by this effectiveness. Failure to identify and mitigate bias in data products can lead to costly errors. By applying supervised machine learning (ML), a neural network can be trained to recognize features in data that a human domain expert would identify as characteristic of poor data quality. In this study, we use magnetotelluric (MT) data as an example of a geophysical data set appropriate for such a training exercise. While MT data are used to estimate the resistivity structure of the subsurface, the concepts we discuss are universal to seismic, potential fields, and other geophysical data sets. We train a neural network, pyMAGIQ (Python-based magnetotelluric impedance qualifier), with multiple hidden layers and demonstrate that it successfully generates the nonlinear mapping function required to assess the quality of MT data. The training set is a large database of frequency-domain MT impedance tensors from the National Science Foundation-funded EarthScope MT project. A human-assigned quality index is associated with each impedance. We apply pyMAGIQ to unrated MT data from the United States and Canada and confirm that the ML-assigned quality factors are consistent with those assigned by trained human operators. We also apply sensitivity analysis to the trained neural network. This reveals that the human- and ML-assigned data quality index depends on the magnitude of the confidence limits on (1) the phases and (2) the continuity of the apparent resistivities and phases with respect to frequency.

This study developed a rapid rice yield estimation workflow and customized yield prediction model by integrating remote sensing and meteorological data with machine learning (ML). Several issues need to be addressed while developing a crop yield estimation model, including data quality issues, data processing issues, selecting a suitable machine learning model that can learn from few available time-series data, and understanding the non-linear relationship between historical crop yield and remote sensing and meteorological factors. This study applied a series of data processing techniques and a customized ML model to improve the accuracy of crop yield estimation at the district level in Nepal. It was found that remote sensing-derived NDVI product alone was not sufficient for accurate estimation of crop yield. After incorporating other meteorological variables into the ML models, estimation accuracy improved dramatically. Along with NDVI, the meteorological variables of rainfall, soil moisture, and evapotranspiration also exhibited a strong association with rice yield. This study also found that stacking multiple tree-based regression models together could achieve better accuracy than benchmark linear regression or standalone ML models. Due to the unique and distinct physio-geographical setting of each district, a variation in estimation accuracy from district to district could be observed. Our data processing and ML model workflow achieved an average of 92% accuracy of yield estimation with RMSE 328.06 kg/ha and MAE 317.21 kg/ha. This methodological workflow can be replicated in other study areas and the results can help the local authorities and stakeholders understand the factors affecting crop yields as well as estimating crop yield before harvesting season to ensure food security and sustainability.

In microwave breast imaging, image reconstruction involves tomographic reconstruction, radar-based imaging, or some combination of both methods (J. F. Deprez, et al., PIER B, 42, 381-403, 2012). With radar-based techniques, similar basic processing steps, i.e., calibration and skin subtraction are taken to process the raw data prior to their deployment in Kirchhoff summation (image focusing). Usually, minimal consideration is given to improve the temporal resolution of the raw signals. However, the spatial resolution associated with the reconstructed image is a function of the temporal resolution of raw data. With simple breast phantoms, i.e., homogeneous breast volume with a single inclusion, an image with low spatial resolution may give a relatively accurate estimate of inclusion location, however lacks sensitivities to the changes in its size. With complicated breast phantoms, i.e., heterogeneous breast volume with multiple malignant and benign inclusions, low imaging resolution may blur malignant inclusions into adjacent features. Inspired by seismic imaging data processing workflow, we propose the method of source wavelet deconvolution for pre-processing prior to Kirchhoff summation. The idea is to improve the temporal resolution of calibrated raw data by compressing the source wavelet. We also propose the method of picking the first break (arrival) of wave propagation for the purpose of locating the scatterer in the signal. Previously, the signals were integrated prior to Kirchhoff summation, which assumes that a local maximum indicates the round-trip distance between the antenna and the scatterer. In reality, this assumption is not always true due to the complexity of wave propagation (A. Ishimaru, 1997). In this sense, the proposed new processing workflow can be described as improved calibration, skin subtraction, deconvolution, and reconstruction.

Summary Conventional seismic data acquisition surveys require a firing time delay from each source to avoid energy overlaps between all the different shot records. But recent developments in continuous seismic recording systems and data separation workflows have allowed us to simultaneously acquire seismic data from different sources at the same time, thereby, greatly reducing operations costs and improving the data acquisition efficiency. However, the simultaneous source approach imposes new challenges in our data processing workflows to separate (deblend) the signal from each source. Various proposed source separation technologies are available to deblend the data, but most of these technologies rely upon coherent noise attenuation techniques that produce results with a significant amount of residual signal from the seismic interference. In addition to the quality limitations, coherent noise attenuation techniques require a substantial amount of additional time to create the best processing workflow that suits the needs of a particular dataset. In this abstract, we present an alternative approach to deblend the High Point 3D broadband long-offset simultaneous-source towed-streamer data using an innovative algorithm that progressively models the source-separated signal while safely eliminating the interference from the data, and provides significantly better results with better source-separation performance.

We have developed a novel framework for combining physics-based forward models and neural networks to advance seismic processing and inversion algorithms. Migration is an effective tool in seismic data processing and imaging. Over the years, the scope of these algorithms has broadened; today, migration is a central step in the seismic data processing workflow. However, no single migration technique is suitable for all kinds of data and all styles of acquisition. There is always a compromise on the accuracy, cost, and flexibility of these algorithms. On the other hand, machine-learning algorithms and artificial intelligence methods have been found immensely successful in applications in which big data are available. The applicability of these algorithms is being extensively investigated in scientific disciplines such as exploration geophysics with the goal of reducing exploration and development costs. In this context, we have used a special kind of unsupervised recurrent neural network and its variants, Hopfield neural networks and the Boltzmann machine, to solve the problems of Kirchhoff and reverse time migrations. We use the network to migrate seismic data in a least-squares sense using simulated annealing to globally optimize the cost function of the neural network. The weights and biases of the neural network are derived from the physics-based forward models that are used to generate seismic data. The optimal configuration of the neural network after training corresponds to the minimum energy of the network and thus gives the reflectivity solution of the migration problem. Using synthetic examples, we determine that (1) Hopfield neural networks are fast and efficient and (2) they provide reflectivity images with mitigated migration artifacts and improved spatial resolution. Specifically, the presented approach minimizes the artifacts that arise from limited aperture, low subsurface illumination, coarse sampling, and gaps in the data.

The UK Catalysis Hub (UKCH) is designing a virtual research environment to support data processing and analysis, the Catalysis Research Workbench (CRW). The development of this platform requires identifying the processing and analysis needs of the UKCH members and mapping them to potential solutions. This paper presents a proposal for a demonstrator to analyse the use of scientific workflows for large scale data processing. The demonstrator provides a concrete target to promote further discussion of the processing and analysis needs of the UKCH community. In this paper, we will discuss the main requirements for data processing elicited and the proposed adaptations that will be incorporated in the design of the CRW and how to integrate the proposed solutions with existing practices of the UKCH. The demonstrator has been used in discussion with researchers and in presentations to the UKCH community, generating increased interest and motivating further development.