A New Feedforward Neural Network Structural Learning Algorithm—Augmentation by Training With Residuals

High pressure air compressor valve fault diagnosis using feedforward neural networks

A hybrid algorithm for Bayesian network structure learning with application to multi-label learning

One of the challenges practitioners face when applying structure learning algorithms to their data involves determining a set of hyperparameters; otherwise, a set of hyperparameter defaults is assumed. The optimal hyperparameter configuration often depends on multiple factors, including the size and density of the usually unknown underlying true graph, the sample size of the input data, and the structure learning algorithm. We propose a novel hyperparameter tuning method, called the Out-of-sample Tuning for Structure Learning (OTSL), that employs out-of-sample and resampling strategies to estimate the optimal hyperparameter configuration for structure learning, given the input dataset and structure learning algorithm. Synthetic experiments show that employing OTSL to tune the hyperparameters of hybrid and score-based structure learning algorithms leads to improvements in graphical accuracy compared to the state-of-the-art. We also illustrate the applicability of this approach to real datasets from different disciplines.

In order to solve the problems that the intelligence algorithm falls into the local optimum easily and has a slow convergence in Bayesian networks (BN) structure learning, an algorithm based on adaptive selection operator with simulated annealing is proposed. This chapter conducts the adaptive selection rule in combination with conditional independence tests of BN nodes to guide the generation of neighbor. In order to better compare the adaptive effect, an algorithm based on selection operator with simulated annealing (SOSA) is proposed; at the same time 15 data sets in the three typical networks are accessed as learning samples. The results of the Bayesian Dirichlet (BD) score, Hamming distance (HD), and evolution time of the network after learning show that it has the quicker convergence and it searches the optimal solution more easily compared with simulated annealing (SA) and SOSA.

This paper proposes a step-by-step technique for combining basic models that forecast electricity consumption in an artificial neural network by the method of preliminary selection and further hybridization. The reported experiments were conducted using data on hourly electricity consumption at the metallurgical plant AO ArcelorMittal Temirtau in the period from January 1, 2019, to November 30, 2021. The current research is related to the planned introduction of a balancing electricity market. 96 combinations of basic models were compiled, differing in the type of neural network, the set of initial data, the order of lag, the learning algorithm, and the number of neurons in the hidden layer. It has been determined that the NARX-type network is the most optimal architecture to forecast electricity consumption. Based on experimental studies, the number of hidden neurons needed to form a planned daily profile should equal 3 or 4; it is recommended to use the conjugate gradient method as a learning algorithm. When selecting models from three groups, it was revealed that the conjugate gradient method produces better results compared to the Levenberg-Marquardt algorithm. It is determined that the values of the selected RMSE error indicator take values of 23.17, 22.54, and 22.56, respectively, for the first, second, and third data groups. The adaptive hybridization method has been shown to reduce the RMSE error rate to 21.73. However, the weights of the best models with values of 0.327 for the first group of data, and 0.336 for the second and third ones, show that the individual use of a separate combination of models is also applicable. The devised forecasting electricity consumption model can be integrated into an automated electricity metering system

Traditional power distribution networks suffer from a lack of real-time observability. This complicates development and implementation of new smart-grid technologies, such as those related to demand response, outage detection and management, and improved load monitoring. In this paper, inspired by proliferation of metering technology, we discuss topology estimation problems in structurally loopy but operationally radial distribution grids from measurements, for example, voltage data, which are either already available or can be made available with a relatively minor investment. The primary objective of this paper is to learn the operational layout of the grid. Further, the structure learning algorithm is extended to cases with missing data, where available observations are limited to a fraction of the grid nodes. The algorithms are computationally efficient—polynomial in time—which is proven theoretically and illustrated in numerical experiments on a number of test cases. The techniques developed can be applied to detect line failures in real time as well as to understand the scope of possible adversarial attacks on the grid.

An important benchmark for closed-loop flow control is the suppression of the von Karman vortex street in the wake of a circular cylinder at a Reynolds number of 100. A low-dimensional Proper Orthogonal Decomposition (POD) is applied to the flow field and a four sensor configuration is placed based on the intensity of the resulting spatial Eigenfunctions. This effort focuses on the performance of a non-linear Artificial Neural Network Estimator (ANNE) used for real-time mapping of flow field sensors to POD states. This investigation examines the sensitivity of ANNE by varying the number of neurons in the hidden layer. The aim is to determine if there is an optimum number of neurons in the hidden layer. Three input data sets were studied, the first had no noise on the training and testing data, the second had 10% noise applied to the training and testing data, and the third had 25% noise applied to the training and testing data sets. In all cases 50 randomly selected training data sets were applied to the back-propagating network. A separate data set was used for testing the feed-forward network. During the validation process, network outputs were compared to desired outputs and the Root Mean Squared (RMS) error was calculated for each of the four output nodes. Results show that in general, the more nodes in the hidden layer the better the performance of the network. However, there appeared to be a consistently reduced error when eight (8) nodes were used in the hidden layer. Results from ANNE when compared to the state-of-the-art Linear Stochastic Estimator (LSE) shows significant benefits.

The radial basis function (RBF) network is a type of neural network that uses a radial basis function as its activation function (Ou, Oyang & Chen, 2005). Because of the better approximation capabilities, simpler network structure and faster learning speed, the RBF networks have attracted considerable attention in many science and engineering field. Horng (2010) used the RBF for multiple classifications of supraspinatus ultrasonic images. Korurek & Dogan (2010) used the RBF networks for ECG beat classifications. Wu, Warwick, Jonathan, Burgess, Pan & Aziz (2010) applied the RBF networks for prediction of Parkinson’s disease tremor onset. Feng & Chou (2011) use the RBF network for prediction of the financial time series data. In spite of the fact that the RBF network can effectively be applied, however, the number of neurons in the hidden layer of RBF network always affects the network complexity and the generalizing capabilities of the network. If the number of neurons of the hidden layer is insufficient, the learning of RBF network fails to correct convergence, however, the neuron number is too high, the resulting over-learning situation may occur. Furthermore, the position of center of the each neuron of hidden layer and the spread parameter of its activation function also affect the network performance considerably. The determination of three parameters that are the number of neuron, the center position of each neuron and its spread parameter of activation function in the hidden layer is very important.

Learning large network (with hundreds of variables) is gaining interest of many researchers with the emergence of high-throughput biological data sources such as micro-array data. In this paper, we investigated the two popular large scale network structure learning algorithms, sparse candidate hill climbing (SCHC) and Grow-Shrinkage(GS) algorithm. The experiments show that in fact both of them have serious effectiveness problems when the number of variables(genes) is large compared to the number of instances(experimental conditions), which is a common case in micro-array data. We further propose a new large scale structure learning algorithm based on Lasso regression. Theoretical analysis in [10] suggested that the L1-norm in lasso regression could make our algorithm especially suitable in the cases that the number of variables and instances is unbalance. Our algorithm achieves much better results than SCHC and GS on the synthetic data. We also show the effectiveness of our algorithm by learning genetic regulatory network modules from a real micro-array data (with more than 6000 genes), combined with the genome-wide location analysis data. The learned results are consistent well with biological knowledge.

This paper investigates the error of determining the coordinates of an acoustic signal source depending on various parameters of the developed software-mathematical models for training a neural network. The authors determined the optimal values of these parameters according to the criterion of minimum error. The study found that the optimal amount of data for training a neural network is 25,000 training pairs, and the number of hidden layers should not exceed 5. Among several forms of sensor arrangement, the best is the square, as it provides a smaller error and uniform filling of the microphone’s perimeter. Their number should not exceed 9, since the best algorithm for training the neural network trainlm requires significant computing resources of the system on which training is performed. An increase in the number of sensors leads to an increase in the training time of the neural network but does not lead to an improvement in accuracy. When studying the effect of the distance to the acoustic signal source and the algorithm for selecting the base sensor for calculating the time difference, it was found that changing these parameters does not affect the accuracy of the neural network. The best structure of the number of hidden neurons in each hidden layer is an expanding structure, in which the number of neurons in each layer increases from the input layer to the middle-hidden layer, then decreases to the output layer. Using the optimized parameters, the neural network was trained and its accuracy was evaluated on two data sets: those used for training and those unknown to the neural network. In both cases, the neural network demonstrated good results. According to the results of the study, it was found that the error of the optimised system is 6.7681 × 10-5 m in the Y coordinate and 6.6120 × 10-5 m in the X coordinate.

Constructing the structure of protein signaling networks by Bayesian network technology is a key issue in the field of bioinformatics. The primitive structure learning algorithms of the Bayesian network take no account of the causal relationships between variables, which is unfortunately important in the application of protein signaling networks. In addition, as a combinatorial optimization problem with a large searching space, the computational complexities of the structure learning algorithms are unsurprisingly high. Therefore, in this paper, the causal directions between any two variables are calculated first and stored in a graph matrix as one of the constraints of structure learning. A continuous optimization problem is constructed next by using the fitting losses of the corresponding structure equations as the target, and the directed acyclic prior is used as another constraint at the same time. Finally, a pruning procedure is developed to keep the result of the continuous optimization problem sparse. Experiments show that the proposed method improves the structure of the Bayesian network compared with the existing methods on both the artificial data and the real data, meanwhile, the computational burdens are also reduced significantly.

This paper presents a versatile hyper-ellipsoidal basis function for function approximation in a given high dimensional space. This hyper-ellipsoidal basis function can be translated and rotated to cover the data based upon the distribution of data in a given high dimensional space. Based on this function, we propose a one-pass hyper-ellipsoidal learning algorithm for which any new incoming data can be fed for learning without involving the previously learned one. This learning algorithm is used to adjust the parameters of the versatile hyper-ellipsoidal basis function. In addition, we propose the hyper-ellipsoidal basis function (HEBF) neural network that uses the one-pass hyper-ellipsoidal neural learning algorithm. The structure of this neural network is similar to the radial basis function (RBF) neural networks. The hidden neurons in the HEBF neural network can be increased or decreased during learning process. The number of the hidden neurons in the network can be grown based on geometric growth criterion and can be reduced by merging the two hidden neurons into a new hidden neuron based on merging criterion during learning process. The merging process can be done independently without considering the learned data set.KeywordsRadial Basis FunctionHide NeuronRadial Basis Function Neural NetworkHigh Dimensional SpaceInitial WidthThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Introduction Atmospheric corrosion is the main cause of metallic heritage degradation when they are exposed to relative humidity, temperature cycle and air contaminants. Corrosivity maps are one of the preventive strategies for atmospheric corrosion mitigation, but they do not provide enough information about corrosion mechanism, besides that long experimentation periods are required [1]. Moreover, electrochemical techniques are alternatives to speed up results, but their relationship with constantly changing environmental variables is difficult to control.Currently, Artificial Intelligence has been introduced in corrosion engineering studies, because of its capability to make predictions based on big data. Also, experimental data of Electrochemical Impedance Spectroscopy had been used in Artificial Neural Network training to obtain Nyquist Diagrams [2]. The objective of this work is to obtain computational models of Artificial Neural Networks to predict atmospheric corrosion in Bronze with nanostructured patina with SiO2, in marine environments. Method Environmental parameters were taken from database previously reported [3,4], and from Civil Protection Secretariat of Veracruz. Whilst electrochemical evaluation consists in Polarization Resistance and Electrochemical Impedance Spectroscopy; they were carried out in a Gamry Interface 1000 Potentiostat with a Saturated Calomel Reference Electrode (SCE) and a graphite bar as counter electrode in an agar gelled cell. Meanwhile, working electrode was bronze with nanostructured patina with SiO2. Two patina were prepared, the first with CuSO4 solution (0.015 mol L-1) and the second applying a CuNO3 solution (20% wt).The 13 variables used in the database for Artificial Neural Networks (ANN) training were: exposure time, chloride concentration, sulfur compounds concentration, relative humidity, precipitation level, windspeed, temperature, nanocoating presence, corrosion potential, corrosion rate, frequency, real and imaginary component of impedance. The values of each variable were treated with measures of central tendency and dispersion, as well as correlation matrix and histograms, which allowed to find a relationship between variables, and choosing the inputs for ANN. The prediction models consisted in 5, 9, 10 and 12 neurons in the input layer (R5E, R9E, R10E and R12E, respectively), a hidden layer, and Zima as only neuron in the output layer. The number of neurons in hidden layer varied from 1 to 8 until the highest coefficient of determination was achieved. The computational model was performed using the toolbox in MATLAB with a feedforward ANN. It is worth mention that the hidden and output layers used a hyperbolic tangent and a linear transfer function, respectively, and a Levenberg-Marquardt algorithm as training function was used in the models. Results and Conclusions Figure 1 presents the coefficient of determination (R2) and root mean squared error (RMSE) with different ANN architectures. In the four models, the highest value was reached with 8 neurons in the hidden layer. Then, the linear regressions for R5E, R9E, R10E and R12E models and 8 neurons in the input layer were compared in Figure 2. In that sense, the Nyquist Diagrams in Figure 3 were simulated with the R9E and R12E models. Also, they were compared with experimental Nyquist Diagrams for Bronze with patina of CuSO4 and CuNO3, with and without SiO2 nanoparticles, at 56 days of exposure in marine atmospheres. The simulations demonstrated how an ANN model with 12 neurons in the input layer, 8 neurons in the hidden layer and 1 neuron in the output layer can be used as a corrosion prediction model of bronze with nanostructured patina with SiO2.

Ensuring fairness when training Machine Learning (ML) models remains a critical challenge, particularly when biases are embedded in the underlying data. This paper presents a fairness-aware graph structure learning framework demonstrating how learning fair graphs leads to fairer data for ML training and, consequently, fairer Artificial Intelligence (AI) decisioning based on such models. Our method incorporates a fairness regularization term into score-based structure learning algorithms, guiding the search towards graph structures that minimize discriminatory pathways while preserving statistical relationships. The learned fair graph structures enable the generation of synthetic datasets with mitigated biases, which can be used to train diverse ML models. This modification is non-trivial, as structure learning algorithms rely on local search strategies, while fairness is a global property that depends on the entire graph structure. Our framework is highly adaptable, compatible with various structure learning algorithms, and seamlessly incorporates different fairness metrics to meet specific contextual needs. Extensive experiments on both real-world and synthetic datasets demonstrate that our approach significantly improves fairness while maintaining competitive predictive performance, offering an interpretable and versatile solution for mitigating bias in AI systems..

Learning the structure of continuous-time Bayesian networks directly from data has traditionally been performed using score-based structure learning algorithms. Only recently has a constraint-based method been proposed, proving to be more suitable under specific settings, as in modelling systems with variables having more than two states. As a result, studying diverse structure learning algorithms is essential to learn the most appropriate models according to data characteristics and task-related priorities, such as learning speed or accuracy. This article proposes alternative algorithms for learning multidimensional continuous-time Bayesian network classifiers, introducing, for the first time, constraint-based and hybrid algorithms for these models. Nevertheless, these contributions also apply to the simpler one-dimensional classification problem for which only score-based solutions exist in the literature. More specifically, the aforementioned constraint-based structure learning algorithm is first adapted to the supervised classification setting. Then, a novel algorithm of this kind, specifically tailored for the multidimensional classification problem, is presented to improve the learning times for the induction of multidimensional classifiers. Finally, a hybrid algorithm is introduced, attempting to combine the strengths of the score- and constraint-based approaches. Experiments with synthetic and real-world data are performed not only to validate the capabilities of the proposed algorithms but also to conduct a comparative study of the available competitors.