Stochastic Gradient Estimation for Artificial Neural Networks

Variants of feedforward backpropagation (BP) artificial neural network (ANNs) such as pseudo Mac Laurin power series order 1 (McL1BP), order 2 (McL2BP), order 3 (McL3BP) and order 4 (McL4BP) models are used to fill in mean annual streamflows. The baseline of modeling process is the standard feedforward backpropagation ANN (StandBP). The sigmoid function is used as activation function. Performance comparisons of data infilling models (ANNs) are conducted using the Root Mean Square Error of Predictions (RMSEp) and graphical plots. To test the performance of the data infilling techniques, selected streamflow gauges (i.e. the Diepkloof (control) gauge on the Wonderboomspruit River and the Molteno (target) gauge on Stormbergspruit River) of the Orange drainage river systems of South Africa have been used. The results demonstrated that relatively higher order ANN models; i.e. McL3BP, McL4BP can still outperform the rest of techniques for 7 % (except StandBP), 20 % and 30 % missing data proportions at Molteno gauge. At the same time they have been outperformed by the rest of ANN models at 13 %. Generally higher and low order Pseudo MacLaurin ANNs are acceptable to fill in missing mean annual flows at Molteno gauge. The accuracy of estimated mean annual values at Molteno is substantially negatively affected beyond 20 % gap size for all ANNs. Hence gap size beyond 20 % yields to relatively higher value of RMSEp for both high and low MacLaurin feedforward ANN techniques. A linear relationship could describe accuracy of estimated values and gap size at Molteno. Further work should include other data regimes as well as other South African catchment areas. Other activation functions should also be tested.

In this work, we proposed a generalized likelihood ratio method capable of training the artificial neural networks with more flexibility: (a)training with discrete activation and loss functions, while the traditional back propagation method cannot train the artificial neural networks with such activations and loss; (b)involving neuronal noises during training and prediction, which will improve the freedom of the model and make it more adaptable to the real environment, especially when environmental noises exist. Numerical results show that the robustness of various artificial neural networks trained by the new method is significantly improved when the input data is affected by both the natural noises and adversarial attacks.

Solving Large Scale Classification Problems with Stochastic Based Optimization

Selection of the most relevant features for training of an artificial neural network (ANN) is of significant importance affecting largely the complexity and accuracy of the trained network. Moreover, deciding the topology and architecture of an ANN including the training algorithm, number of hidden layers, activation functions and neurons in each layer is challenging for the effectiveness of an ANN to solve a particular problem. In this paper, we present a novel strategy to result into an optimized topological architecture for an ANN and selection method for the most important experimental features available for training. The proposed method is found to efficiently design and model flows in solar collectors and subsequently estimating pressure loss coefficients for flows in the tee junctions vastly used in solar collectors with high accuracy. Experimental results demonstrate that the number of features used in training of ANN can be reduced up to 85% by identifying the features upon which the pressure loss coefficients’ estimation shows strong dependency. Performance analysis of various training algorithms has been carried out with variation in hidden layers, activation functions and neurons that results into minimum mean square error leading to minimal percent relative error.

Fast and accurate diagnosis is very important for kidney disease. This research conducts and analyzes by using Nguyen Widrow Algorithm in Back Propagation method in artificial neural network for kidney disease diagnosis with the aim to improve the accuracy in predicting and time efficiency in diagnosing. The Nguyen Widrow algorithm is very capable of accelerating convergence and stabilizing the learning process in artificial neural networks, which is also expected to present a meaningful contribution to the handling of health data. This study uses MATLAB as a platform for algorithm implementation and a dataset of medical records of kidney disease patients collected from a hospital that specializes in treating kidney disease patients. The data pre-processing and artificial neural network modeling stages use the Nguyen Widrow algorithm, while the model training process uses the Back Propagation method. The results showed that the Nguyen Widrow algorithm was able to improve the accuracy of predicting someone suffering from kidney disease compared to using only the Back Propagation method. Analysis of the performance of the model shows a significant improvement in stability and convergence speed during the learning process. This indicates that data processing and medical decision making becomes more efficient. On the other hand, this research also studied the challenges and limitations that will be faced in terms of implementation of the Nguyen Widrow algorithm. Also the sensitivity of the initialization parameters, the need for the quality of the dataset to be used in training the model.This research reveals the ability of the Nguyen Widrow algorithm to improve the performance of artificial neural networks in diagnosing kidney disease. By implementing this algorithm in MATLAB, the results show that the use of the latest data processing technology and analysis tools can provide significant improvements in accuracy and efficiency in the medical field. In addition, this research is expected to provide a new direction in the development of machine learning algorithms for applications in the healthcare field, especially for diagnosing kidney disease. By further utilizing this technology, it contributes significantly to improving the quality of healthcare and treatment outcomes for patients suffering from kidney disease.

This research aims to recognize the pattern of pulmonary disease on x-ray radiography image using artificial neural network (ANN) method. The images, which were used such as images of healthy pulmonary, pulmonary tuberculosis, and pulmonary tumour. Pattern recognition was using an extraction feature of GLCM (Gray Level Co-occurrence Matrix) and back propagation method. Before being identified, the images were processed by median filter and adaptive histogram equalization. The GLCM features that used were homogeneity, energy, contrast, variance and correlation. The parameters were learning rate and hidden layer. Learning rate was 0.3 and hidden layer was 25. Back propagation training showed 100% accuracy, which all of 44 images were used had been successfully identified. From the result, the healthy pulmonary showed 60% accuracy, 83.3% for pulmonary tuberculosis and 100% for pulmonary tumor. Hence, the overall result showed 81.25% accuracy, which 13 of 16 images had been successfully identified. From these result, extraction feature of GLCM using back propagation method was capable to recognize the pattern of pulmonary disease. However, due to narrow range of the feature, this application may not be used optimally for comparing features in every category of images. Therefore, the further research is needed to determine the best features and parameters of training back propagation.

Artificial neural networks (ANNs) have gained significant popularity in the last decade for solving narrow AI problems in domains such as healthcare, transportation, and defense. As ANNs become more ubiquitous, it is imperative to understand their associated safety, security, and privacy vulnerabilities. Recently, it has been shown that ANNs are susceptible to a number of adversarial evasion attacks - inputs that cause the ANN to make high-confidence misclassifications despite being almost indistinguishable from the data used to train and test the network. This work explores to what degree finding these examples may be aided by using side-channel information, specifically switching power consumption, of hardware implementations of ANNs. A black-box threat scenario is assumed, where an attacker has access to the ANN hardware’s input, outputs, and topology, but the trained model parameters are unknown. Then, a surrogate model is trained to have similar functional (i.e. input-output mapping) and switching power characteristics as the oracle (black-box) model. Our results indicate that the inclusion of power consumption data increases the fidelity of the model extraction by up to 30% based on a mean square error comparison of the oracle and surrogate weights. However, transferability of adversarial examples from the surrogate to the oracle model was not significantly affected.

A new architecture selection method based on tabu search for artificial neural networks

این مطالعه به منظور بررسی پیش بینی شاخص های عملکرد تولیدی در مرغان تخم گذار با استفاده از شبکه های عصبی مصنوعی و رگرسیون غیرخطی چندگانه انجام شد. بررسی بر روی اطلاعات چهار دوره متوالی پرورش در یک واحد پرورش مرغ تخم گذار صورت گرفت. روش های داده-کاوی شامل رگرسیون خطی و غیر خطی، شبکه عصبی پرسپترون سه لایه، شبکه عصبی پرسپترون چهار لایه و شبکه عصبی با تابع پایه ای شعاعی بود. در این مدل ها از متغیرهای سن گله، میزان خوراک مصرفی و فصل تولید به عنوان متغیر پیشگو و شاخص های عملکرد تولیدی شامل درصد تخم-گذاری، وزن توده ای تخم مرغ تولیدی و ضریب تبدیل غذایی به عنوان متغیر پاسخ استفاده شد. نتایج نهایی رگرسیون های خطی نشان داد که برای تمامی متغیرهای وابسته مورد مطالعه متغیر مستقل سن گله معنی دار می باشد. بنابراین رگرسیون غیر خطی شاخص های عملکرد تولیدی در مقابل سن برای مقایسه با شبکه های عصبی مختلف مورد بررسی قرار گرفت و برای مقایسه کلیه مدل ها از ضریب تعیین (R2) و میانگین قدر مطلق خطا (MAE) استفاده شد. نتایج نشان داد بین شبکه های عصبی مصنوعی مختلف مورد مطالعه، شبکه با تابع پایه ای شعاعی بهتر از سایر مدل های در پیش‌بینی شاخص های عملکرد تولیدی مرغان تخم گذار عمل می کند.

A New Multilevel Input Layer Artificial Neural Network for Predicting Flight Delays at JFK Airport

Artificial Neural Network (ANN) Is Superior to the Pre-Endoscopic Rockall Score in Predicting Which Patients Will Benefit From Urgent EGD in Acute Upper GI Bleeding (UGIB)

Spiking neural networks (SNNs) more closely mimic the human brain than artificial neural networks (ANNs). For SNNs, time-to-first-spike (TTFS) encoding, which represents the output values of neurons based on the timing of a single spike, has been proposed as a promising model to reduce power consumption. Adversarial attacks that can lead ANNs to misrecognize images have been reported in many studies. However, the characteristics of TTFS-based SNNs trained using a backpropagation algorithm against adversarial attacks have not yet been clarified. In particular, the dependence of the robustness against adversarial attacks on spike timings has not been investigated. In this brief, we investigated the robustness of SNNs against adversarial attacks and compared it with that of an ANN. We found that SNNs trained with the appropriate temporal penalty settings are more robust against adversarial images than ANNs.

An artificial neural network (ANN) is a network designed with adaptation to a computer system. The developed computer system will perform functions oriented to the way the brain works (neuron concept). This study is an extension to the study of the suitability of ANN to be applied in numbers of applications, especially in the field of medical engineering. ANN has been widely being used in medicine, ANN is widely applied in education, research, and even decision making. In this study, ANN will be trained for pre-testing to predict the cardiac abnormalities symptom based on selected reference parameters. This reference parameter is better known as the input parameter to the ANN to detect cardiac abnormalities, among which are the of the height of peak/wave (amplitude) and time occurrence of peak/wave (duration of time) extracted from the electrocardiogram (ECG) signal. A complete ECG complex contains a P peak, a QRS wave, and a T peak. For each P peak, QRS wave, and T peak, amplitude height and duration will be measured to serve as input parameters. This makes six parameters defined as inputs to the ANN. This study has used a Multilayer Perceptron (MLP) network as ANN structure by being trained using three different training algorithms namely Backpropagation (BP), Lavenberg Marquardt (LM) and Bayesian Regularization (BR). At the end of the study, it showed the MLP network which by BR training algorithm gave the highest accuracy prediction (94.04%), followed by LM (92.95%) and BP (88.77%). In this study all MLP networks were activated using the Tansig activation function.

Swertia chirata is a medicinal plant studied for its ability to synthesize polyshaped gold nanoparticles (AuNP). The process of AuNP biosynthesis was studied using artificial neural networks (ANN) with different activation function on output node (logistic or linear) and different training algorithm (back propagation or Levenberg–Marquardt). The maximum biosynthesis was checked under the optimized condition of 17.24% leaf extract, pH 4.61, gold chloride concentration 4 mM and temperature 53.61 °C. A significant improvement in the model efficiency for predicting AuNP biosynthesis around 37.60%, in terms of root mean square error was obtained with the developed ANN-linear2 model, compared to the traditional response surface methodology.

This paper proposes a technique to improve Artificial Neural Network (ANN) prediction accuracy using Particle Swarm Optimization (PSO) combiner. A hybrid system consists of two stages with the first stage containing two ANNs. The first ANN predictor is a multi-layer feed-forward network trained with error back-propagation and the second predictor is a functional link network. These two predictors are combined in the second stage using PSO combiner in a linear and non-linear fashion. The proposed method is applied to problem of predicting daily natural gas consumption. The performance of ANN predictors and combination methods are tested on real data from four different gas utilities. The experimental results show that the proposed particle swarm optimization combiners results in more accurate prediction compared to using single ANN predictor. Prediction accuracy improvement of the proposed PSO combiners have been shown using hypothesis testing.