Evaluation of compression member buckling and post-buckling behavior using artificial neural network

<abstract><p>The training of artificial neural networks (ANNs) with rectified linear unit (ReLU) activation via gradient descent (GD) type optimization schemes is nowadays a common industrially relevant procedure. GD type optimization schemes can be regarded as temporal discretization methods for the gradient flow (GF) differential equations associated to the considered optimization problem and, in view of this, it seems to be a natural direction of research to <italic>first aim to develop a mathematical convergence theory for time-continuous GF differential equations</italic> and, thereafter, to aim to extend such a time-continuous convergence theory to implementable time-discrete GD type optimization methods. In this article we establish two basic results for GF differential equations in the training of fully-connected feedforward ANNs with one hidden layer and ReLU activation. In the first main result of this article we establish in the training of such ANNs under the assumption that the probability distribution of the input data of the considered supervised learning problem is absolutely continuous with a bounded density function that every GF differential equation admits for every initial value a solution which is also unique among a suitable class of solutions. In the second main result of this article we prove in the training of such ANNs under the assumption that the target function and the density function of the probability distribution of the input data are piecewise polynomial that every non-divergent GF trajectory converges with an appropriate rate of convergence to a critical point and that the risk of the non-divergent GF trajectory converges with rate 1 to the risk of the critical point. We establish this result by proving that the considered risk function is <italic>semialgebraic</italic> and, consequently, satisfies the <italic>Kurdyka-Łojasiewicz inequality</italic>, which allows us to show convergence of every non-divergent GF trajectory.</p></abstract>

This article, written by Technology Editor Dennis Denney, contains highlights of paper SPE 94357, "Treating Uncertainties in Reservoir-Performance Prediction With Neural Networks," by J.P. Lechner, SPE, OMV A.G., and G. Zangl, SPE, Schlumberger Information Solutions, prepared for the 2005 SPE Europec/EAGE Annual Conference, Madrid, Spain, 13–16 June. In development projects, reservoir parameters are known only within certain ranges, which results in various realizations of the subsurface. Because of the computational time involved, simulation models to obtain a probability distribution of possible outcomes cannot cover all possible parameter combinations. Creating a response surface based on a reduced number of simulation runs becomes necessary. Such a response surface can be used to approximate results for several variations of input parameters. An approach in which reservoir response is captured by an artificial neural network (ANN) has been investigated. The trained ANN model was used in Monte Carlo simulations to generate the probability distribution of possible outcomes. Introduction Often, simulation models are built to design field-development plans. Because of the low level of information during the early development stage, a range of uncertainty in the prediction scenarios must be considered. Such uncertainties can be handled with ANNs. Numerical models (simulation models) are detailed and powerful predictive tools that can answer many questions regarding subsurface engineering. However, especially in the early development phase, uncertainties are large, and prediction results will span a broad range. To take into account the whole range of possible outcomes of reservoir simulation, optimization routines would have to run the numerical models perhaps thousands of times. Although the most likely solution that respects subsurface uncertainties is of interest, the most profitable solution for reservoir management must be found. Because of the computational time involved, these methodologies often are not used. Building a response surface that can predict many possible outcomes of a numerical simulation by processing a large variety of input parameters could provide a solution to this problem. The response surface must cover the non-linear dependencies between input and output parameters. Experimental design and response-surface modeling has been used to delineate subsurface uncertainty. Common applications include determining the probability distribution of original oil in place from geological models, quantifying uncertainty in predevelopment or preredevelopment projects, predicting production performance from different realizations of a geological model, and optimizing the location of new wells to maximize net profit. Methodology Fig. 1 shows the proposed workflow. As a first step, a limited number of simulation runs is carried out to define the most sensitive input parameters. In these runs, parameters are altered one at a time, either to the minimum or to the maximum. After selecting the most influential input parameters, three values (minimum, best-estimate, and maximum) are defined, and an experimental design is set up. The experimental design aims at obtaining the maximum information with a minimum number of simulation runs. However, practice showed that a minimum number of training data sets are required by the ANN to be able to make sufficiently accurate predictions. With these data sets containing varying input parameters and the corresponding output, an ANN is trained and tested with several additional simulation runs. After defining the probability distribution of the input parameters, the trained ANN is used as a proxy function for a Monte Carlo simulation. Thereby, a probability distribution of the desired output parameter (e.g., cumulative production after a certain number of years) is gained. The quality of the ANN is measured by the root-mean-square error of the deviation between the predicted and the actual output.

This paper presents a novel technique for the supervised training of feed-forward artificial neural networks (ANN) using the Harmony Search (HS) algorithm. HS is a stochastic meta-heuristic that is inspired from the improvisation process of musicians. Unlike Backpropagation, HS is non-trajectory driven. By modifying an existing improved version of HS & adopting a suitable ANN data representation, we propose a training technique where two of HS probabilistic parameters are determined dynamically based on the best-to-worst (BtW) harmony ratio in the current harmony memory instead of the improvisation count. This would be more suitable for ANN training since parameters and termination would depend on the quality of the attained solution. We have empirically tested and verified our technique by training an ANN with a benchmarking problem. In terms of overall training time and recognition, our results have revealed that our method is superior to both the original improved HS and standard Backpropagation.

The main aim of this study is to estimate the bearing strength on pin loaded composite plates using test data of different ambient conditions depending on time. Artificial neural network (ANN) tool is used for prediction purpose. Artificial neural network program is developed by MATLAB software. The composite plates were divided into nine groups, each one contains 60 specimens. These groups were kept in different environment conditions. To obtain the optimum geometrical dimensions of specimens, the distance from hole axis to the side of the test sample of the pin diameter ratio (E/D) and the sample width to pin diameter ratio (W/D) were changed systematically in the experimental samples. Data from fatigue test results obtained from the multi-layered, feedforward and backpropagation algorithm were used to train the artificial neural network model. In modeling of artificial neural network, the test conditions, the test periods, the day intervals, the distance from hole axis to the side of test sample of the pin diameter ratio (E/D) and the sample width to pin diameter ratio (W/D) were used as input parameters and the bearing strength data was used as output parameter. The values obtained from the artificial neural network training and testing were evaluated by applying statistical analyses that are widely used in ANN models. It is widely known that difficult experimental studies and complexity of the analytical expression could be solved by ANN models as it was seen in many studies. This study has also shown that artificial neural network is a convenient method for predicting the bearing strength on pin loaded composite plates.

The aim of this article is to use artificial neural networks (ANN) to perform the structural design of confined masonry buildings. ANN is easy to operate and allows to reduce the time and cost of seismic designs. To generate the artificial neural network, training models (traditional confined masonry designs) are used to identify the input and output parameters. From this, the final architecture and activation functions are defined for each layer of the ANN. Finally, ANN training is carried out using the "backpropagation" algorithm to obtain the matrix of weights and thresholds that allow the network to operate and provide preliminary structural designs with a 10% margin of error, with respect to the traditional design, in the dimensions and reinforcements of the structural elements.

BACKGROUND: Currently, artificial intelligence in the form of artificial neural networks is being actively implemented in a number of areas of our lives, including medicine. In particular, in otorhinolaryngology, artificial neural networks are used to analyze images obtained during endoscopic examinations of patients (e.g., videolaryngoscopy) [1–3]. The interpretation of laryngoscopic images often presents significant difficulties for practicing physicians, which reduces the frequency of detection of precancerous laryngeal diseases and contributes to the increase in the number of patients with stage III–IV laryngeal cancer [4, 5]. This underscores the significance of prompt performance and accurate interpretation of the findings of endoscopic examinations of patients with laryngeal disorders. Artificial neural networks can be employed to analyze the results of videolaryngoscopy, furnishing the physician with supplementary information that can enhance diagnostic accuracy and diminish the probability of error [6, 7]. AIM: The study aims to develop and train an artificial neural network for recognizing characteristic features of laryngeal neoplasms and variants of laryngeal normality. MATERIALS AND METHODS: The study was conducted under the grant of the Moscow Center for Innovative Technologies in Healthcare (grant No. 2112-1/22) entitled “Using Neural Networks (Artificial Intelligence Algorithms) for Control and Improving the Quality of Diagnosis and Treatment of Diseases of Laryngeal and Ear Structures through Digital Technologies”.The following methods were used during the course of the study: data collection for the creation of a photobank (dataset) of medical images obtained during videolaryngoscopy; data partitioning for the formation of datasets for individual nosologies and groups of diseases; the method of consilium; analysis of the accuracy of recognition and classification of digital endoscopic images; and training of classification neural networks. Consequently, a dataset comprising 1,471 laryngeal images in digital formats (JPEG, BMP) was assembled, labelled, and uploaded for the purpose of training the artificial neural network. Of the total number of images, 410 were classified as pertaining to laryngeal formation, while 1061 were classified as variants of normality. Subsequently, the neural network was trained and tested to identify the signs of normal and laryngeal masses. RESULTS: The results of the testing of the artificial neural network indicated the formation of an inaccuracy matrix, the calculation of the value of recognition accuracy, the calculation of the quality indicators of the model performance, and the construction of the ROC curve. The developed and trained artificial neural network demonstrated an accuracy of 86% in recognizing the signs of laryngeal masses and norms. CONCLUSIONS: This study demonstrates that a trained artificial neural network can successfully distinguish between signs of normal and laryngeal masses in endoscopic photographs. With further training of the neural network and achievement of high accuracy, this technology can be used in clinical practice as an assistant in the interpretation of laryngoscopic images and early diagnosis of laryngeal masses. It can also be employed to control and improve the quality of diagnosis and treatment of diseases of the throat, nose, and ears by primary care physicians.

The effects of data preprocessing in the determination of coagulant dosing rate

In Masonry buildings, walls are the main structural elements resistant to lateral loads. Reinforcing masonry walls by shotcreting one- or two-sides is one of the most common reinforcement methods. Considering the economic losses and casualties caused by damage to these elements in previous earthquakes, it is necessary to investigate the seismic behavior of these walls. The existing data can train artificial neural networks, and the behaviors could be generalized for future cases. In this study, the prediction of cyclic behavior parameters of reinforced masonry walls with one- and two-way shotcrete is investigated using different artificial neural network methods. Input parameters of the neural network include length, thickness, height, mortar shear strength, mortar compressive strength, mesh type, spring dimensions, rebar diameter, average thickness of shotcrete, and concrete compressive strength. Output parameters were the relative displacement of yield, relative displacement corresponding to maximum resistance, final relative displacement, yield strength, displacement resistance corresponding to maximum resistance, ultimate resistance, and initial stiffness. The results showed that the feed-forward back-propagation neural network could accurately predict the examined output parameters compared to other models, which can be considered as an alternative to some time-consuming and costly laboratory and analytical investigations.

In this paper, we show the possibility of creating and identifying the features of an artificial neural network (ANN), which consists of mathematical models of biological neurons. The FitzHugh–Nagumo (FHN) system is used as a paradigmatic model demonstrating basic neuron activities. First, in order to reveal how biological neurons can be embedded within an ANN, we train the ANN with nonlinear neurons to solve a basic image recognition problem with an MNIST database; next, we describe how FHN systems can be introduced into this trained ANN. After all, we show that an ANN with FHN systems inside can be successfully trained with improved accuracy comparing with first trained ANN and then with inserted FHN systems. This approach opens up great opportunities in terms of the direction of analog neural networks, in which artificial neurons can be replaced by more appropriate biological ones.

Looking at the artificial neural networks’ literature, most of the studies started with feedforward artificial neural networks and the training of many feedforward artificial neural networks models are performed with derivative-based algorithms such as levenberg–marquardt and back-propagation learning algorithms in the first studies. In recent years, although many new heuristic algorithms have been proposed for different aims these heuristic algorithms are also frequently used in the training process of many different artificial neural network models. Pi-sigma artificial neural networks have different importance than many artificial neural network models with its higher-order network structure and superior forecasting performance. In this study, the training of Pi-Sigma artificial neural networks is performed by differential evolution algorithm uses DE/rand/1 mutation strategy. The performance of the proposed method is evaluated by two data sets and seen that the proposed method has a very effective performance compared with many artificial neural network models.

1176 OUTCOME PREDICTION IN PATIENTS WITH BLADDER CANCER AFTER RADICAL CYSTECTOMY USING ARTIFICIAL NEURAL NETWORKS

Joint A-SNN: Joint training of artificial and spiking neural networks via self-Distillation and weight factorization

In this work a modification was made on the algorithm of Artificial Neural Networks (NN) Training of the Multilayer Perceptron type (MLP) based on multi-objective optimization (MOBJ), to increase its computational efficiency. Usually, the number of efficient solutions to be generated is a parameter that must be provided by the user. In this work, this number is automatically determined by an algorithm, through the usage of golden section, being generally less when specified, showing a sensible reduction in the processing time and keeping the high generalization capability of the obtained solution from the original method.

In ischemic stroke, the extent of ischemic lesion recovery is one of the most important correlate of functional recovery in brain. Using a set of acute phase MR images (Diffusion-Weighted - DWI, T 1 -Weighted - T1WI, T 2 -Weighted T2WI, and proton density weighted - PDWI) for inputs, and the chronic T2WI at 3 months as an outcome measure, an Artificial Neural Network (ANN) was trained to predict the 3-month outcome in the form of a pixel-by-pixel forecast of the chronic T2WI. The ANN was trained and tested using 14 slices from 3 subjects using a K-Folding Cross-Validation (KFCV) method with 14 folds. The Area Under the Receiver Operator Characteristic Curve (AUROC) for 14 folds was used for training, testing and optimization of the ANN. After training and optimization, the ANN produced a map that was well correlated (r = 0.88, p ≪ 0.0001) with the T2WI at 3 months. To confirm that the trained ANN performed well against a new dataset, 13 slices from 4 other patients were shown to the trained ANN. The prediction made by the ANN had an excellent overall performance (AUROC = 0.82), and was very well correlated to the 3-month ischemic lesion on T2-Weighted image.

Multilayer artificial neural networks (ANN) are often used for the solution of classification problems or for the time series forecasting. An appropriate number of learning and testing patterns must be available for the ANN training. Each training pattern is composed of n input parameters and m output parameters. Number m is usually given by the problem formulation, but the number n may be often selected from a greater set of input parameters. An optimal selection of input parameters is a very important task especially in a situation when the number of usable input parameters is great and the analytical relations between the input and output parameters are not known. The number of neurons in all ANN layers must be generally kept as small as possible because of the optimal generalisation ability. In this paper we present a possible way for the selection of significant input parameters (the so called markers), which are the most important ones from the point of view of influence on the output parameters. These parameters are later used for the training of ANN. A statistical approach is usually used for this reason [5]. After some experience in the ANN application we recognised that the approach based on mathematical logic, i. e. the GUHA method (General Unary Hypotheses Automaton) is also suitable for the determination of markers. Besides the minimisation of the number of elements in the input layer of ANN, also the number of neurons in hidden layers must be optimised. For this reason standard methods of pruning can be used, described e. g. in [1]. We have used this method in the following applications: - Optimisation of the intervals between the major overhaul of plane engines by the analysis of tribodiagnostic data. Only selected types of chemical pollution in oil can be taken into account. - Prediction of bleeding of patients with chronic lymphoblastic leukemia. Only a part of parameters about the patient is important from this point of view (see [2]). - Optimisation of the quality and reliability prediction of artificial resin production in chemical factory. Only a part of the production parameters (times of production phases, temperatures, percentage of components etc.) have straight influence on the product. - Optimisation of the prosody control in the text-to-speech synthesis. This application is described in the paper.