Optimized algorithm for multipoint geostatistical facies modeling based on a deep feedforward neural network

We study the eigenvalue distribution of random matrices pertinent to the analysis of deep neural networks. The matrices resemble the product of the sample covariance matrices, however, an important difference is that the analog of the population covariance matrix is now a function of random data matrices (synaptic weight matrices in the deep neural network terminology). The problem has been treated in recent work [J. Pennington, S. Schoenholz and S. Ganguli, The emergence of spectral universality in deep networks, Proc. Mach. Learn. Res. 84 (2018) 1924–1932, arXiv:1802.09979] by using the techniques of free probability theory. Since, however, free probability theory deals with population covariance matrices which are independent of the data matrices, its applicability in this case has to be justified. The justification has been given in [L. Pastur, On random matrices arising in deep neural networks: Gaussian case, Pure Appl. Funct. Anal. (2020), in press, arXiv:2001.06188] for Gaussian data matrices with independent entries, a standard analytical model of free probability, by using a version of the techniques of random matrix theory. In this paper, we use another version of the techniques to extend the results of [L. Pastur, On random matrices arising in deep neural networks: Gaussian case, Pure Appl. Funct. Anal. (2020), in press, arXiv:2001.06188] to the case where the entries of the data matrices are just independent identically distributed random variables with zero mean and finite fourth moment. This, in particular, justifies the mean field approximation in the infinite width limit for the deep untrained neural networks and the property of the macroscopic universality of random matrix theory in this case.

Seismic inversion, a nonlinear, non-unique-solution, and human-prone error problem, seeks the optimal solution in multivariate space, a task that AI algorithms perform better. So, inversion with AI becomes appropriate to extract information from post-stack data. A probabilistic neural network (PNN) and deep feedforward neural network (DFNN) inverted seismic data using multiattributes, and their results were compared with those provided by a model-based inversion (MBI). Applied to a post-stack volume of the Frontier Formation at the Teapot-Dome field, Wyoming-USA, the three methods provided porosity volumes (ϕ), gamma rays (GR), and Lamé impedances (µρ, λρ). The comparison between well-logs and the provided volumes revealed a reliable PNN prediction, while DFNN achieved the best geologically correlated parameter sections and a less reliable MBI prediction. The geomorphological time-slice analysis discriminated high ϕ-µρ areas and low GR–λρ areas, geologically interpreted as a deltaic environment, with distributary channels and beachfront lobes subject to the action of the tides, which agrees with the well-log interpretation and the basin geological evolution. The results enhanced the stacked section’s importance by the geological information contained, which would contribute to reducing the prospecting risk. Finally, PNN and DFNN encouraging results reaffirm their best searching-in-multivariate space ability.

Abstract. Time series data in practical applications always contain missing values due to sensor malfunction, network failure, outliers etc. In order to handle missing values in time series, as well as the lack of considering temporal properties in machine learning models, we propose a spatiotemporal prediction framework based on missing value processing algorithms and deep recurrent neural network (DRNN). By using missing tag and missing interval to represent time series patterns, we implement three different missing value fixing algorithms, which are further incorporated into deep neural network that consists of LSTM (Long Short-term Memory) layers and fully connected layers. Real-world air quality and meteorological datasets (Jingjinji area, China) are used for model training and testing. Deep feed forward neural networks (DFNN) and gradient boosting decision trees (GBDT) are trained as baseline models against the proposed DRNN. Performances of three missing value fixing algorithms, as well as different machine learning models are evaluated and analysed. Experiments show that the proposed DRNN framework outperforms both DFNN and GBDT, therefore validating the capacity of the proposed framework. Our results also provides useful insights for better understanding of different strategies that handle missing values.

As an important modeling parameter in multi-point geostatistics, training images determine the modeling effect to a great extent. It is necessary to evaluate and optimize the applicability of candidate training images before modeling by multi-point geostatistics. Conventional two-dimensional training images can’t describe the overlapping relation of sedimentary facies in space, and there is a deficiency in describing the event relation of single data. This paper puts forward a new training image optimization method. The basic idea is to arrange and analyze sand bodies filled with sedimentary sand bodies in point dams and river channels in different periods. The method of obtaining three-dimensional training images is to use sand thickness maps and sedimentary facies maps for spatial constraints. The simulation test shows that compared with the sedimentary facies model obtained from two-dimensional training images, the sedimentary facies model obtained from three-dimensional training images through multi-point geostatistics has high compatibility and is more in line with geological understanding. On the basis of fully understanding the development characteristics of the sedimentary system and quantitative geometry of sedimentary body in the study area, sedimentary microfacies models based on sequential indicator simulation method and target simulation method are established, respectively. By comparing and analyzing the sedimentary microfacies models established by three different methods, the results show that the multi-point geostatistics can stably present the planar distribution characteristics and overlapping spatial relationship of sedimentary microfacies and reproduce the complex spatial structure and geometric shape of fluvial facies. The model established by this method is more in line with the geological sedimentary model.

Dynamic properties, such as shear modulus and damping ratio, are critical for civil engineering applications and essential for accurate dynamic response analysis. This study introduces a novel Deep Neural Network (DNN) approach to predict the normalized shear modulus (G/Gmax ) and damping ratio (D) of granular soils over a wide strain range. Utilising a comprehensive dataset from cyclic triaxial (CT) and resonant column (RC) tests, we developed a Deep Feed-Forward Neural Network (DFFNN) model. The model incorporates grading characteristics, shear strain, void ratio, mean effective confining pressure, consolidation stress ratio, and specimen preparation method as inputs. The DFFNN demonstrated high accuracy with testing results of 0.9830 for G/Gmax and 0.9396 for D, outperforming traditional empirical models and other intelligent techniques such as Shallow Neural Network (SNN), Support Vector Regression (SVR), and Gradient Boosting Regression (GBR). This data-driven approach offers a robust and adaptable method for predicting the dynamic properties of granular soils across diverse conditions.

Contingency analysis (CA) is a vital technique of assessing power system resilience during outages by detecting overloads, voltage issues, and generator faults, strengthening grid security and emergency response planning. For the Ethiopian Electric Power (EEP) network, an N‑1 contingency analysis was conducted on Extra High Voltage (EHV) transmission lines. A dual‑depth Deep Feed Forward Neural Network (DFFNN) was employed on MATLAB/MATPOWER platform to predict bus voltage magnitudes and active power flows across branches. The DFFNN achieved strong training performance, with Mean Square Error (MSE) values of 3.887 x 10<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-5</sup> and 7.32 x10<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-3</sup>, alongside high regression values of 0.99234 and 0.99479, confirming its predictive accuracy. These predictions were used to implement a novel Hybrid Performance Index (PI<sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">hyb</sub>), which ranks contingencies according to severity. Results revealed that failure of either circuit of the Dedessa–Holeta 500 kV double-line, sourced from the Great Ethiopian Renaissance Dam (GERD), was the most severe contingency. This event produced a PI<sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">hyb</sub> of 1.798, causing voltage instability at 19 buses and severe overloading of the Dedessa–Nekemte 132 kV line. Subsequent critical contingencies were also linked to GERD, with PI<sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">hyb</sub> values of 0.473 and 0.321. Corrective and preventive measures are recommended to mitigate these risks, ensuring protection against cascading failures that could ultimately lead to a total blackout.

Purpose: The aim of the article is to develop an algorithm for classifying cracks in the analyzed images using modern methods of deep machine learning and transfer learning based on pretrained convolutional neural network - Inception-ResNet-v2. Design/Methodology/Approach: Transfer learning based on the pretrained convolutional neural network was used to categorize the images. The fully conected layer of the Inception-ResNet-v2 network has been modified. The last layer was trained using a two-class (binary) linear SVM (Support Vector Machine). In total, 20,000 training cases (images) were used to train the fully connected layer within transfer learning process. The research analyzed the possibility of using the deep neural networks for quick and fully automatic identification of cracks / defects on the surface of analyzed parts. Findings: The results indicate that pretrained convolutional neural network using SVM to train a fully connected layer is a very effective solution for visual crack / fault detection. In the analyzed model, a positive classification was obtained at the level of 99.89%. Practical Implications: The model presented in the article can be used in quality control carried out by process monitoring. An effective model for identifying defective parts can be used in both logistics and production processes. Originality/Value: A novelty is the use of a freely available, deep neural network trained to classify 1000 categories of various images for binary categorization of faults (cracks). The algorithm was adjusted by replacing the primary, 1000-output fully connected layer in the Inception-ResNet-v2 network with a binary layer (2 categories). The fully connected layer has been trained using the classification version of the popular SVM learner, but thanks to the combination of this layer with the sophisticated fearure extraction ability of the pre-trained Inception-ResNet-v2 deep network, the resulting predictive model enables the classification of defects with a very high level of accuracy.

Deep neural networks are effective tools for solving actual tasks such as data mining, modeling, forecasting, pattern recognition, clustering, classification etc. They differ with respect to the architecture design, learning methods and so on. Most simple and widely used are deep feed-forward supervised NNs.

Review spam (fake review) detection is increasingly important taking into consideration the rapid growth of internet purchases. Therefore, sophisticated spam filters must be designed to tackle the problem. Traditional machine learning algorithms use review content and other features to detect review spam. However, as demonstrated in related studies, the linguistic context of words may be of particular importance for text categorization. In order to enhance the performance of review spam detection, we propose a novel content-based approach that considers both bag-of-words and word context. More precisely, our approach utilizes n-grams and the skip-gram word embedding method to build a vector model. As a result, high-dimensional feature representation is generated. To handle the representation and classify the review spam accurately, a deep feed-forward neural network is used in the second step. To verify our approach, we use two hotel review datasets, including positive and negative reviews. We show that the proposed detection system outperforms other popular algorithms for review spam detection in terms of accuracy and area under ROC. Importantly, the system provides balanced performance on both classes, legitimate and spam, irrespective of review polarity.

Accurate segmentation of the retinal microvasculature is a critical step in the quantitative analysis of the retinal circulation, which can be an important marker in evaluating the severity of retinal diseases. As manual segmentation remains the gold standard for segmentation of optical coherence tomography angiography (OCT-A) images, we present a method for automating the segmentation of OCT-A images using deep neural networks (DNNs). Eighty OCT-A images of the foveal region in 12 eyes from 6 healthy volunteers were acquired using a prototype OCT-A system and subsequently manually segmented. The automated segmentation of the blood vessels in the OCT-A images was then performed by classifying each pixel into vessel or nonvessel class using deep convolutional neural networks. When the automated results were compared against the manual segmentation results, a maximum mean accuracy of 0.83 was obtained. When the automated results were compared with inter and intrarater accuracies, the automated results were shown to be comparable to the human raters suggesting that segmentation using DNNs is comparable to a second manual rater. As manually segmenting the retinal microvasculature is a tedious task, having a reliable automated output such as automated segmentation by DNNs, is an important step in creating an automated output.

Automated image classification is an essential task of the computer vision field. The tagging of images into a set of predefined groups is referred to as image classification. The implementation of computer vision to automate image classification would be beneficial because manual image evaluation and identification can be time-consuming, particularly when there are many images of different classes. Deep learning approaches are proven to overperform existing machine learning techniques in many fields in recent years, and computer vision is one of the most notable examples. The very deep neural network (VDCNN) is a powerful deep learning model for image classification, and this paper examines it briefly using MNIST handwritten digit dataset. This dataset is used to prove the efficacy of very deep neural networks over other deep learning models. The proposed study aims to comprehend the very deep neural network architecture used to accomplish a handwritten digit recognition task. The feasibility of the proposed model is evaluated using mean accuracy, validation accuracy, and standard deviation. The study results of the very deep neural network model are compared to a convolutional neural network and convolutional neural network with batch normalization. According to the results of the comparison study, very deep neural networks achieve high accuracy of 99.1% for a handwritten dataset. The outcome of the proposed work is used to interpret how well a very deep neural network performs when compared to the other two models of deep neural networks. This proposed architecture may be used to automate the classification of handwritten digits dataset.

On neural networks application in integral sliding mode control

Video hyperlinking represents a classical example of multimodal problems. Common approaches to such problems are early fusion of the initial modalities and crossmodal translation from one modality to the other. Recently, deep neural networks, especially deep autoencoders, have proven promising both for crossmodal translation and for early fusion via multimodal embedding. A particular architecture, bidirectional symmetrical deep neural networks, have been proven to yield improved multimodal embeddings over classical autoencoders, while also being able to perform crossmodal translation. In this work, we focus firstly at evaluating good single-modal continuous representations both for textual and for visual information. Word2Vec and paragraph vectors are evaluated for representing collections of words, such as parts of automatic transcripts and multiple visual concepts, while different deep convolutional neural networks are evaluated for directly embedding visual information, avoiding the creation of visual concepts. Secondly, we evaluate methods for multimodal fusion and crossmodal translation, with different single-modal pairs, in the task of video hyperlinking. Bidirectional (symmetrical) deep neural networks were shown to successfully tackle downsides of multimodal autoencoders and yield a superior multimodal representation. In this work, we extensively tests them in different settings, with different single-modal representations, within the context of video-hyperlinking. Our novel bidirectional symmetrical deep neural networks are compared to classical autoencoders and are shown to yield significantly improved multimodal embeddings that significantly (alpha=0.0001) outperform multimodal embeddings obtained by deep autoencoders with an absolute improvement in precision at 10 of 14.1% when embedding visual concepts and automatic transcripts and an absolute improvement of 4.3% when embedding automatic transcripts with features obtained with very deep convolutional neural networks, yielding 80% of precision at 10.

The raising number of elderly people urges the research of systems able to monitor and support people inside their domestic environment. An automatic system capturing data about the position of a person in the house, through accelerometers and RGBd cameras can monitor the person activities and produce outputs associating the movements to a given tasks or predicting the set of activities that will be executes. We considered, for the task the classification of the activities a Deep Convolutional Neural Network. We compared two different deep network and analyzed their outputs.

Electronic properties prediction enhancement of 36 ternary III-IB-VI alloys using a deep feed-forward neural network