Research on Rules Extraction from Neural Network based on Linear Insertion

Artificial neural networks (ANNs) are systems which are derived from the field of neuroscience and are characterized by intensive arithmetic operations. These networks display interesting features such as parallelism, classification, optimization, adaptation, generalization and associative memories. Since the McCulloch and Pitts pioneering work (McCulloch, W.S., & Pitts, W. (1943), there has been much discussion on the topic of ANNs implementation, and a huge diversity of ANNs has been designed (C. Lindsey & T. Lindblad, 1994). The benefits of using such implementations is well discussed in a paper by R. Lippmann (Richard P. Lipmann, 1984): “The great interest of building neural networks remains in the high speed processing that can be achieved through massively parallel implementation”. In another paper Clark S. Lindsey (C.S Lindsey, Th. Lindbald, 1995) posed a real dilemma of hardware implementation: “Built a general, but probably expensive system that can be reprogrammed for several kinds of tasks like CNAPS for example? Or build a specialized chip to do one thing but very quickly, like the IBM ZISC Processor”. To overcome this dilemma, most researchers agree that an ideal solution should relay the performances obtained using specific hardware implementation and the flexibility allowed by software tools and general purpose chips. Since their commercial introduction in the mid- 1980’s, and due to the advances in the development of both of the microelectronic technology and the specific CAD tools, FPGAs devices have progressed in an evolutionary and revolutionary way. The evolution process has allowed faster and bigger FPGAs, better CAD tools and better technical support. The revolution process concerns the introduction of high performances multipliers, Microprocessors and DSP functions. This has a direct incidence to FPGA implementation of ANNs and a lot of research has been carried to investigate the use of FPGAs in ANNs implementation (Amos R. Omandi & Jagath C. rajapakse, 2006). Another attractive key feature of FPGAs is their flexibility, which can be obtained at different levels: exploitation of the programmability of FPGA, dynamic reconfiguration or run time reconfiguration (RTR), (Xilinx XAPP290, 2004) and the application of the design for reuse concept (Keating, Michael; Bricaud, Pierre, 2002). However, a big disadvantage of FPGAs is the low level hardware oriented programming model needed to fully exploit the FPGA’s potential performances. High level based VHDL synthesis tools have been proposed to bridge the gap between the high level application requirements and the low level FPGA hardware but these tools are not algorithmic or application specific. Thus, special concepts need to be developed for automatic ANN implementation before using synthesis tools. In this paper, we present a high level design methodology for ANN implementation that attempts to build a bridge between the synthesis tool and the ANN design requirements. This method offers a high flexibility in the design while achieving speed/area performances constraints. The three implementation figures of the ANN based back propagation algorithm are considered. These are the off-type implementation, the on-chip global implementation and the dynamic reconfiguration choices of the ANN. To achieve our goal, a design for reuse strategy has been applied. To validate our approach, three case studies are considered using the Virtex-II and Virtex-4 FPGA devices. A comparative study is done and new conclusions are given.

Artificial neural networks (ANN) have been successfully applied to a multitude of problems in various fields. One of the most prominent application fields is time series forecasting. Although ANN produces accurate forecasts in many time series implementations, there are still some problems with using ANN. ANN approach is composed of some components which are architecture structure, learning algorithm and activation function. These components have important effect on the forecasting performance of ANN. An important decision is the selection of optimum architecture of neural network that consists of determining the numbers of neurons in the layers of the network. Therefore, various approaches have been proposed to find the best ANN architecture in the literature. However, the most preferred method is still trial and error method for finding a good architecture. In this study, the total health expenditures made by social security institution in Turkey is forecasted by using a hybrid heuristic method proposed by Aladag [10] which is based on feed forward neural networks and Tabu search algorithm. It is seen that the hybrid forecasting approach produces very accurate results.

Artificial neural networks (ANN) is increasingly being used to solve engineering problems. Successful welding operations are at the top of engineering problems. The friction stir welding method, which is used to combine the materials under the melting temperature, is also a method that can be analyzed with ANN. In this study, Al-7075 workpieces (with and without aging treatment) were joined by the friction stir welding method. Tension tests were applied to the joined workpieces and the resulting values were modeled by ANN. In ANN model; The tensile strength of the assembled parts was tried to be estimated by input parameters (the type of the material, the number of revolutions and the feed rate). In the created ANN model; Different nerve numbers, different transfer functions, using different layer numbers, the models created by alternative network structures have been tested. In these experiments, prediction performance was measured using neuron and algorithm type variables. At the end of this study, it was seen that the ANN model for the parts joined with the FSW process made predictions with high consistency. The best estimates were obtained with the Levenberg Marquardt (LM) algorithm and the ANN model established with 3 neurons. This ANN model is seen as a suitable solution for this problem with 98% success.

Artificial Neural Networks (ANNs), as a nonlinear and adaptive information processing systems, play an important role in machine learning, artificial intelligence, and data mining. But the performance of ANNs is sensitive to the number of neurons, and chieving a better network performance and simplifying the network topology are two competing objectives. While Genetic Algorithms (GAs) is a kind of random search algorithm which simulates the nature selection and evolution, which has the advantages of good global search abilities and learning the approximate optimal solution without the gradient information of the error functions. This paper makes a brief survey on ANNs optimization with GAs. Firstly, the basic principles of ANNs and GAs are introduced, by analyzing the advantages and disadvantages of GAs and ANNs, the superiority of using GAs to optimize ANNs is expressed. Secondly, we make a brief survey on the basic theories and algorithms of optimizing the network weights, optimizing the network architecture and optimizing the learning rules, and make a discussion on the latest research progresses. At last, we make a prospect on the development trend of the theory.

Artificial neural networks (ANN) are receiving a lot of attention because of their nonlinear mapping ability in the field of short term load forecast (STLF). ANN based STLF model commonly use back propagation algorithm, that may not converge properly, that affects the forecast accuracy. A hybrid approach, based on artificial neural network (ANN) and genetic algorithm (GA) that combines the advantages of each technique is proposed in this research. Genetic algorithm is implemented for the optimization of the architecture of feedforward neural network and selection of its initial weight values. Error back propagation algorithm for the training of the optimized neural network will be implemented. The second stage of this research is related with the complete training of the neural network based on genetic algorithm, using genetic manipulation of chromosomes. The results show that this approach produced better output in terms of enhanced forecast accuracy.

Artificial neural networks (ANN) are among the nonlinear prediction techniques popular in the last two decades. Recent studies show that ANN can be modeled with different training techniques. ANN is usually trained by the backpropagation method (BP). In this study, ANN structures were trained by using artificial bee colony algorithm (ABC) and, weight and bias values were tried to be determined. ABC training (ANN-ABC) was tested over three different datasets and compared with the BP training (ANN-BP) results. In addition to use ABC in modeling, different error types such as mean square error (MSE), mean absolute percent error (MAPE) and adjusted coefficient of determination (R) have been used in the training. The results on popular time series datasets have shown that ABC based ANN training yields successful results in forecasting.

Artificial Neural Networks (ANNs) have seen an explosion of interest over the last two decades and have been successfully applied in all fields of chemistry and particularly in analytical chemistry. Inspired from biological systems and originated from the perceptron, i.e. a program unit that learns concepts, ANNs are capable of gradual learning over time and modelling extremely complex functions. In addition to the traditional multivariate chemometric techniques, ANNs are often applied for prediction, clustering, classification, modelling of a property, process control, procedural optimisation and/or regression of the obtained data. This paper aims at presenting the most common network architectures such as Multi-layer Perceptrons (MLPs), Radial Basis Function (RBF) and Kohonen's self-organisations maps (SOM). Moreover, back-propagation (BP), the most widespread algorithm used today and its modifications, such as quick-propagation (QP) and Delta-bar-Delta, are also discussed. All architectures correlate input variables to output variables through non-linear, weighted, parameterised functions, called neurons. In addition, various training algorithms have been developed in order to minimise the prediction error made by the network. The applications of ANNs in water analysis and water quality assessment are also reviewed. Most of the ANNs works are focused on modelling and parameters prediction. In the case of water quality assessment, extended predictive models are constructed and optimised, while variables correlation and significance is usually estimated in the framework of the predictive or classifier models. On the contrary, ANNs models are not frequently used for clustering/classification purposes, although they seem to be an effective tool. ANNs proved to be a powerful, yet often complementary, tool for water quality assessment, prediction and classification.

Artificial neural network (ANN) is a non-linear statistical technique that being used to describe material behavior. This paper proposes using artificial neural networks (ANNS) in predicting silicon and nickel recovery. During the experimental work, the used optimum parameters are reaction time is 25 min., temperature is 950 ᵒC, Ni2O3 /Al wt. (weight) ratio is 0.082, and Na2SiF6 / Al wt. ratio is 1. Some tests such as chemical analysis, microstructure examination (EDX mapping), XRD diffraction were carried out on the produced alloys. The obtained experimental results are used to train the artificial neural network (ANN). While reaction time, temperature, Ni2O3 /Al wt. ratio, and Na2SiF6 / Al wt. ratio are used as ANN's inputs. Silicon and nickel recovery are used as ANN's outputs. The used ANN consists of three layers; Input layer that includes 4 neurons, the hidden layer includes 9 neurons, while the output layer contains 2 neurons. The Levenberg-Marquardt (LM) is used as the training function. Optimal mean square errors (MSE) for the ANN during predicting and estimating silicon and nickel recovery equal 0.0358, 0.0034, respectively, when reaction time is the variable and other parameters are kept constant, MSE equal 1.4007e-04, 1.3478e-04 when temperature is variable and other parameters are kept constant, MSE equal 1.3839e-04, 9.9891e-05 when Ni2O3/Al wt. ratio was the variable and other parameters are kept constant and finally MSE equal 0.0287, 0.0263 when Na2SiF6 / Al wt. ratio is variable and other parameters are kept constant.

Neural Networks with Asymptotics Control

Artificial neural network (ANN) analysis was used to predict the permeability of selected compounds through Caco-2 cell monolayers. Previously reported models, which were shown to be useful in the prediction of permeability values, use many structural parameters. More complex equations have also been proposed using both linear and non-linear relationships, including ANN analysis and various structural parameters. But proposed models still need to be developed using different neuron patterns for more precise predictions and a better understanding of which factors affect the permeation. To develop a simple and useful model or method for easy prediction is also a general need. Permeability coefficients (log kp) were obtained from various literature sources. Some structural parameters were calculated using computer programs. Multiple linear regression analysis (MLRA) was used to predict Caco-2 cell permeability for the set of 50 compounds (r2 = 0.403). A successful ANN model was developed, and the ANN produced log kp values that correlated well with the experimental ones (r2 = 0.952). The permeability of a compound, famotidine, which has not previously been studied, through the Caco-2 cell monolayer was investigated, and its permeability coefficient determined. It was then possible to compare the experimental data with that predicted using the trained ANN with previously determined Caco-2 cell permeability values and structural parameters of compounds. The model was also tested using literature values. The developed and described ANN model in this publication does not require any experimental parameters; it could potentially provide useful and precise prediction of permeability for new drugs or other penetrants.

Artificial Neural Network (ANN) model along with Back Propagation Algorithm (BPA) has been applied in many fields, especially in hydrology and water resources management to simulate or forecast rainfall runoff process, discharge and water level - time series, and other hydrological variables. Several researches have recently been focusing to compare the applicability of ANN model with other theory-driven and data-driven approaches. The comparison of ANN with M5 model trees for rainfall-runoff forecasting, with ARMAX models for deriving flow series, with AR models and regression models for forecasting and estimating daily river flows have been carried out. The better results that were implemented by ANN model have been concluded. So, this research trend is continued for the comparison of ANN model with Tank, Harmonic, Thomas and Fiering models in simulation of the monthly runoffs at Dong Nai river basin, Viet Nam. The results proved ANN being the best choice among these models, if suitable and enough data sources were available.

Artificial Neural Network (ANN) based tap changer control of closed primary bus and cross network connected parallel transformers has demonstrated potential use in power distribution system. In those research works the proposed ANN for application in this control were developed using various algorithms and concluded that a network trained by Bayesian Regularization (BR) backpropagation algorithm produced the best performance measured in terms of correct tap changing decisions. However, further improvement of ANN based transformer tap changer operation is always desirable. A general rule for obtaining good generalization is to use the smallest network that solves the problem. In this paper, we show that a small sized ANN is obtainable for further improvement of transformer tap changer operation by modifying the standard Cascade-Correlation algorithm. The modification incorporates weight smoothing of output layer weights in Cascade-Correlation learning using Bayesian frame work. Experimental results demonstrate that significant improvement in performance is achieved when an ANN is trained by modified Cascade-Correlation algorithm instead of standard Cascade-Correlation or Bayesian Regularization backpropagation algorithm. A comparison of performances of different algorithms in application to transformer tap changer operation is analyzed and the results are presented.

Artificial neural network (ANN) model was used for predicting colour properties of 100% cotton denim fabrics, including colour yield (in terms of K/S value) and CIE L*, a*, b*, C*, and h° values, under the influence of cellulase treatment with various combinations of cellulase processing parameters. Variables examined in the ANN model included treatment temperature, treatment time, pH, mechanical agitation, and fabric yarn twist level. The ANN model was compared with a linear regression model where the ANN model produced superior results in the prediction of colour properties of cellulase-treated 100% cotton denim fabrics. The relative importance of the examined factors influencing colour properties was also investigated. The analysis revealed that cellulase treatment processing parameters played an important role in affecting the colour properties of the treated 100% denim cotton fabrics.

Artificial neural network (ANN) can play a vital role in its application to on-load tap changer of distribution transformers since the input variation in practical field is enormous due to continuous changes in the power system conditions and neural network is capable of handling this variation due to its robust generalization capability. In this paper we use the two algorithms, namely, scaled conjugate gradient (SCG) and Bayesian regularization (BR) for training an ANN to control the automatic on load tap changer of two parallel transformers connected across the power network. Primarily five data sets are constructed considering voltage phase angle differences between the sources feeding the primary windings while the secondaries are connected in paralleling mode. For each data set, accuracy in ANN based tap changer decision is investigated and BR algorithm yields better results than SCG algorithm. In the second stage, another combined data set is formed by analyzing the trend of false responses in primary results. Results show significant improvement in performance when combined data set is used for training. With this data set, difference in performance between BR and SCG algorithms reduces, however BR still performs better with significantly high accuracy in tap changer operation. The results demonstrate the potential applicability of ANN in tap changer operation of parallel transformers.

Artificial neural network (ANN) is increasingly applied in clinical medicine. We therefore constructed an ANN to predict intracellular water (ICW) volume in 44 healthy Taiwaners. Demographic and anthropometric data were recorded as predictors, and ICW volume measured by bioelectrical impedance analysis (ICW-BIA) was the reference. ICW volume predicted by ANN (ICW-ANN) was compared with ICW-BIA. ICW-BIA (21.26 +/- 0.58 l) and ICW-ANN (21.25 +/- 0.57 l) was insignificantly different (p = 0.76). ICW-BIA and ICW-ANN were strongly correlated (r = 0.94, p < 0.0001) with a significant agreement (mean difference, 0.01; lower and upper limits of agreement, -2.31 and 2.33) in Bland-Altman plot. Passing-Bablok regression was described as ICW-BIA = 1.04 x ICW-ANN - 0.49, with 95% confidence interval for slope 0.94-1.14 and for intercept -2.76-1.49, indicating that both methods were interchangeable. ANN provided an excellent alternative of BIA to predict ICW volume in healthy subjects.