Efficient self-organizing multilayer neural network for nonlinear system modeling

The paper is focused on the idea to demonstrate the advantages of deep learning approaches over ordinary shallow neural network on their comparative applications to image classifying from such popular benchmark databases as FERET and MNIST. An autoassociative neural network is used as a standalone program realized the nonlinear principal component analysis for prior extracting the most informative features of input data for neural networks to be compared further as classifiers. A special study of the optimal choice of activation function and the normalization transformation of input data allows to improve efficiency of the autoassociative program. One more study devoted to denoising properties of this program demonstrates its high efficiency even on noisy data. Three types of neural networks are compared: feed-forward neural net with one hidden layer, deep network with several hidden layers and deep belief network with several pretraining layers realized restricted Boltzmann machine. The number of hidden layer and the number of hidden neurons in them were chosen by cross-validation procedure to keep balance between number of layers and hidden neurons and classification efficiency. Results of our comparative study demonstrate the undoubted advantage of deep networks, as well as denoising power of autoencoders. In our work we use both multiprocessor graphic card and cloud services to speed up our calculations. The paper is oriented to specialists in concrete fields of scientific or experimental applications, who have already some knowledge about artificial neural networks, probability theory and numerical methods.

Electric Powered Wheelchair (EPW) used by people with disabilities, who cannot move or with a limited ability to move. The traditional technique that used to control the powered wheelchair can be a challenging and stressful operation. Therefore, there is a demand to find alternative ways to perform this process. In that, the voice recognition is a natural form of pointing that can use to replace the joystick whilst still allowing for similar control. Neural network one of the effective ways to recognize the speech commands, because of its ability to grant a trainable powered wheelchair controller with a convenient way, ease of use and attractiveness. In this situation, the neural networks are providing a trainable system for each person irrespectively of his disability. Smart Electric-Powered Wheelchair (SEPW) is an electric-powered wheelchair based on Artificial Neural Networks (ANN) Technique can recognize the voice commands and replace the traditional control of powered wheelchairs with intelligent technology. So, a neural network for the feed-forward Multi-layer perceptron has been trained to recognize isolated spoken words (commands), such as ‘right, ‘left,' ‘stop' and so on. Initially, the design of the system's network started with one hidden layer using five neurons (according to [1] formula). During the training process, the number of hidden neurons has been increased to keep the mean square error (MSE) of the system minimum as possible. Furthermore, the system structure is designed based on a Multi Layers Perceptron (MLP) neural system for five output neurons, which represent the powered wheelchair's commands

In this paper, a computational intelligent technique genetic algorithm (GA) is implemented for the optimization of artificial neural network (ANN) architecture. The network structures are normally selected on the basis of the developer's prior knowledge or hit and trial approach is used for this purpose. ANN based models are frequently used for the prediction of future load, because of their learning and mapping ability to address the non linear nature of electrical load. The proposed technique provides a pathway to determine the best ANN architecture, prior to the training and learning process of neural network. Multi-objective algorithm is proposed in this research which optimizes the ANN architecture that leads to enhancement in load forecast accuracy and reduction in the computational cost. The results of several experiment conducted during this work, exhibits that forecast accuracy is considerably enhanced by using an optimized and reduced ANN structure.

Purpose In order to analyze the text complexity of Chinese and foreign academic English writings, the artificial neural network (ANN) under deep learning (DL) is applied to the study of text complexity. Firstly, the research status and existing problems of text complexity are introduced based on DL. Secondly, based on Back Propagation Neural Network (BPNN) algorithm, analyzation is made on the text complexity of Chinese and foreign academic English writings. And the research establishes a BPNN syntactic complexity evaluation system. Thirdly, MATLAB2013b is used for simulation analysis of the model. The proposed model algorithm BPANN is compared with other classical algorithms, and the weight value of each index and the model training effect are further analyzed by statistical methods. Finally, L2 Syntactic Complexity Analyzer (L2SCA) is used to calculate the syntactic complexity of the two libraries, and Mann–Whitney U test is used to compare the syntactic complexity of Chinese English learners and native English speakers. The experimental results show that compared with the shallow neural network, the deep neural network algorithm has more hidden layers and richer features, and better performance of feature extraction. BPNN algorithm shows excellent performance in the training process, and the actual output value is very close to the expected value. Meantime, the error of sample test is analyzed, and it is found that the evaluation error of BPNN algorithm is less than 1.8%, of high accuracy. However, there are significant differences in grammatical complexity among students with different English writing proficiency. Some measurement methods cannot effectively reflect the types and characteristics of written language, or may have a negative relationship with writing quality. In addition, the research also finds that the measurement of syntactic complexity is more sensitive to the language ability of writing. Therefore, BPNN algorithm can effectively analyze the text complexity of academic English writing. The results of the research provide reference for improving the evaluation system of text complexity of academic paper writing. Design/methodology/approach In order to analyze the text complexity of Chinese and foreign academic English writings, the artificial neural network (ANN) under deep learning (DL) is applied to the study of text complexity. Firstly, the research status and existing problems of text complexity are introduced based on DL. Secondly, based on Back Propagation Neural Network (BPNN) algorithm, analyzation is made on the text complexity of Chinese and foreign academic English writings. And the research establishes a BPNN syntactic complexity evaluation system. Thirdly, MATLAB2013b is used for simulation analysis of the model. The proposed model algorithm BPANN is compared with other classical algorithms, and the weight value of each index and the model training effect are further analyzed by statistical methods. Finally, L2 Syntactic Complexity Analyzer (L2SCA) is used to calculate the syntactic complexity of the two libraries, and Mann–Whitney U test is used to compare the syntactic complexity of Chinese English learners and native English speakers. The experimental results show that compared with the shallow neural network, the deep neural network algorithm has more hidden layers and richer features, and better performance of feature extraction. BPNN algorithm shows excellent performance in the training process, and the actual output value is very close to the expected value. Meantime, the error of sample test is analyzed, and it is found that the evaluation error of BPNN algorithm is less than 1.8%, of high accuracy. However, there are significant differences in grammatical complexity among students with different English writing proficiency. Some measurement methods cannot effectively reflect the types and characteristics of written language, or may have a negative relationship with writing quality. In addition, the research also finds that the measurement of syntactic complexity is more sensitive to the language ability of writing. Therefore, BPNN algorithm can effectively analyze the text complexity of academic English writing. The results of the research provide reference for improving the evaluation system of text complexity of academic paper writing. Findings In order to analyze the text complexity of Chinese and foreign academic English writings, the artificial neural network (ANN) under deep learning (DL) is applied to the study of text complexity. Firstly, the research status and existing problems of text complexity are introduced based on DL. Secondly, based on Back Propagation Neural Network (BPNN) algorithm, analyzation is made on the text complexity of Chinese and foreign academic English writings. And the research establishes a BPNN syntactic complexity evaluation system. Thirdly, MATLAB2013b is used for simulation analysis of the model. The proposed model algorithm BPANN is compared with other classical algorithms, and the weight value of each index and the model training effect are further analyzed by statistical methods. Finally, L2 Syntactic Complexity Analyzer (L2SCA) is used to calculate the syntactic complexity of the two libraries, and Mann–Whitney U test is used to compare the syntactic complexity of Chinese English learners and native English speakers. The experimental results show that compared with the shallow neural network, the deep neural network algorithm has more hidden layers and richer features, and better performance of feature extraction. BPNN algorithm shows excellent performance in the training process, and the actual output value is very close to the expected value. Meantime, the error of sample test is analyzed, and it is found that the evaluation error of BPNN algorithm is less than 1.8%, of high accuracy. However, there are significant differences in grammatical complexity among students with different English writing proficiency. Some measurement methods cannot effectively reflect the types and characteristics of written language, or may have a negative relationship with writing quality. In addition, the research also finds that the measurement of syntactic complexity is more sensitive to the language ability of writing. Therefore, BPNN algorithm can effectively analyze the text complexity of academic English writing. The results of the research provide reference for improving the evaluation system of text complexity of academic paper writing. Originality/value In order to analyze the text complexity of Chinese and foreign academic English writings, the artificial neural network (ANN) under deep learning (DL) is applied to the study of text complexity. Firstly, the research status and existing problems of text complexity are introduced based on DL. Secondly, based on Back Propagation Neural Network (BPNN) algorithm, analyzation is made on the text complexity of Chinese and foreign academic English writings. And the research establishes a BPNN syntactic complexity evaluation system. Thirdly, MATLAB2013b is used for simulation analysis of the model. The proposed model algorithm BPANN is compared with other classical algorithms, and the weight value of each index and the model training effect are further analyzed by statistical methods. Finally, L2 Syntactic Complexity Analyzer (L2SCA) is used to calculate the syntactic complexity of the two libraries, and Mann–Whitney U test is used to compare the syntactic complexity of Chinese English learners and native English speakers. The experimental results show that compared with the shallow neural network, the deep neural network algorithm has more hidden layers and richer features, and better performance of feature extraction. BPNN algorithm shows excellent performance in the training process, and the actual output value is very close to the expected value. Meantime, the error of sample test is analyzed, and it is found that the evaluation error of BPNN algorithm is less than 1.8%, of high accuracy. However, there are significant differences in grammatical complexity among students with different English writing proficiency. Some measurement methods cannot effectively reflect the types and characteristics of written language, or may have a negative relationship with writing quality. In addition, the research also finds that the measurement of syntactic complexity is more sensitive to the language ability of writing. Therefore, BPNN algorithm can effectively analyze the text complexity of academic English writing. The results of the research provide reference for improving the evaluation system of text complexity of academic paper writing.

Performance improvement of artificial neural networks designed for safety key parameters prediction in nuclear research reactors

Electrical load forecasting is one of the important parts for smart grid system. The reliable prediction of the load demand contributes to the efficient and economical operations and planning. The artificial neural network is used extensively in load demand forecasting. The nonlinear nature of the electrical load demand conforms to the ability of the artificial neural network in calculating the nonlinear relationship of inputs and outputs. Among many models of neural networks, radial basis neural networks yield superior performance in small error and fast simulation time. However, it is challenge to design the radial basis neural networks. The excessive numbers of hidden neurons lead to lacking of generalization or so called overfitting problems. This paper proposes an approach to design the radial basis neural networks that use as least numbers of hidden neurons as possible. The error criterion is optimized based on modified genetic algorithm as the numbers of hidden neurons are incrementally increased. Simulation results of short term load forecasting are calculated in Matlab, and compared to the orthogonal least square error method. The proposed approach gives better results with the same numbers of hidden neurons.

Accurate estimation of both runoff and sediment yield is required for proper watershed management. Artificial neural network (ANN) models were developed, to predict both runoff and sediment yield on a daily and weekly basis, for a small agricultural watershed. A total of five models were developed for predicting runoff and sediment yield, of which three models were based on a daily interval and the other two were based on a weekly interval. All five models were developed both with one and two hidden layers. Each model was developed with five different network architectures by selecting a different number of hidden neurons. The models were trained using monsoon season (June to October) data of five years (1991–1995) for different sizes of architecture, and then tested with respective rainfall and temperature data of monsoon season (June to October) of two years (1996–1997). Training was conducted using the Levenberg–Marquardt backpropagation where the input and output were presented to the neural network as a series of learning sets. Simulated surface runoff and sediment yield were compared with observed values and the minimum root-mean-square error and Nash Sutcliff efficiency (coefficient of efficiency) criteria were used for selecting the best performing model. Regression models for predicting daily and weekly runoff and sediment yield were also developed using the above training datasets, whereas these models were tested using the testing datasets. In all cases, the ANN models performed better than the linear regression based models. The ANN models with a double hidden layer were observed to be better than those with single hidden layer. Further, the ANN model prediction performance improved with increased number of hidden neurons and input variables. As a result, models considering both rainfall and temperature as input performed better than those considering rainfall alone as input. Training and testing results revealed that the models were predicting the daily and weekly runoff and sediment yield satisfactorily. Therefore, these ANN models based on simple input can be used for estimation of runoff and sediment yield, missing data, and testing the accuracy of other models.

Conclusion. This study demonstrated that the artificial neural network can successfully classify Mandarin Chinese tone patterns produced by multiple children. The neural network can be used as an objective way of evaluating tone production of children. Objectives. Traditionally, tone production is evaluated subjectively using human listeners. The aim of the present study was to investigate the efficacy of using an artificial neural network in evaluating tone production of Mandarin-speaking children. Subjects and methods. Speech materials were recorded from 61 normal-hearing children. The fundamental frequency (F0) of each monosyllabic word was extracted and then used as inputs to a feed-forward backpropagation artificial neural network. The number of inputs was set at 12, whereas the number of hidden neurons was set at 16 in the neural network. The output layer consisted of four neurons representing the four Mandarin tone patterns. The tone recognition performance of the neural network was further compared with that of native Mandarin-speaking adult listeners. Results. The neural network successfully classified the tone patterns of the 61 child speakers with an accuracy of about 85% correct. This high accuracy exceeded the tone recognition performance by the adult listeners. Individual child speakers showed varied tone production accuracy as recognized by the adult listeners or by the neural network.

Visible and near infrared spectroscopy is a non-destructive, green, and rapid technology that can be utilized to estimate the components of interest without conditioning it, as compared with classical analytical methods. The objective of this paper is to compare the performance of artificial neural network (ANN) (a nonlinear model) and principal component regression (PCR) (a linear model) based on visible and shortwave near infrared (VIS-SWNIR) (400-1000 nm) spectra in the non-destructive soluble solids content measurement of an apple. First, we used multiplicative scattering correction to pre-process the spectral data. Second, PCR was applied to estimate the optimal number of input variables. Third, the input variables with an optimal amount were used as the inputs of both multiple linear regression and ANN models. The initial weights and the number of hidden neurons were adjusted to optimize the performance of ANN. Findings suggest that the predictive performance of ANN with two hidden neurons outperforms that of PCR.

A pruning feedforward small-world neural network based on Katz centrality for nonlinear system modeling

Mill tailings at Kolar Gold Fields are creating particulate pollution on air environment. In the previous study, multiple regression models were developed for the prediction of particulate matter concentrations using data of meteoro- logical parameters (wind speed, wind direction, temperature, humidity and solar radiation) and particulate matter (PM10 and TSP) monitored in different seasons(1). Artificial neural network is an excellent predictive and data analysis tool for the evaluation of air pollutants. Therefore, the data were used for the development of neural network models. During develop- ment of models, the values 0.02, 0.5 and 0.7 were used as target error, learning rate and momentum respectively. Three hidden layers were used to obtain acceptable values. Performance of the models was evaluated using those sets of data which were not used during learning of neural network. Architecture of developed networks, number of hidden neurons and weights, normalised and relative error, importance and sensitivity, etc have been discussed in this paper.

این مطالعه به منظور بررسی پیش بینی شاخص های عملکرد تولیدی در مرغان تخم گذار با استفاده از شبکه های عصبی مصنوعی و رگرسیون غیرخطی چندگانه انجام شد. بررسی بر روی اطلاعات چهار دوره متوالی پرورش در یک واحد پرورش مرغ تخم گذار صورت گرفت. روش های داده-کاوی شامل رگرسیون خطی و غیر خطی، شبکه عصبی پرسپترون سه لایه، شبکه عصبی پرسپترون چهار لایه و شبکه عصبی با تابع پایه ای شعاعی بود. در این مدل ها از متغیرهای سن گله، میزان خوراک مصرفی و فصل تولید به عنوان متغیر پیشگو و شاخص های عملکرد تولیدی شامل درصد تخم-گذاری، وزن توده ای تخم مرغ تولیدی و ضریب تبدیل غذایی به عنوان متغیر پاسخ استفاده شد. نتایج نهایی رگرسیون های خطی نشان داد که برای تمامی متغیرهای وابسته مورد مطالعه متغیر مستقل سن گله معنی دار می باشد. بنابراین رگرسیون غیر خطی شاخص های عملکرد تولیدی در مقابل سن برای مقایسه با شبکه های عصبی مختلف مورد بررسی قرار گرفت و برای مقایسه کلیه مدل ها از ضریب تعیین (R2) و میانگین قدر مطلق خطا (MAE) استفاده شد. نتایج نشان داد بین شبکه های عصبی مصنوعی مختلف مورد مطالعه، شبکه با تابع پایه ای شعاعی بهتر از سایر مدل های در پیش‌بینی شاخص های عملکرد تولیدی مرغان تخم گذار عمل می کند.

Biosorption of copper(II) ions by flax meal: Empirical modeling and process optimization by response surface methodology (RSM) and artificial neural network (ANN) simulation

The Sixth International Symposium on Neural Networks (ISNN 2009)

The artificial neural networks (ANN) by their capacities of training, classification, and decision, give a solution to bearing diagnosis problem by the automatic classification of the vibratory signals corresponding to the various states the machines. They are intended to increase the precision(accuracy) and to reduce errors caused by subjective human judgments. However it is important to note that the ANNs in the aids to diagnosis must be set for optimum performance. The non-existence of predefined rules for ANNs parameters setting (number of hidden neurons in each hidden layers etc…) obstruct the achievement of optimal performances. The use of genetic algorithm (GA) can solve this problem by the parameters and structure optimization of ANN. This paper discusses the use of the ANN multilayer Perceptron (MLP), for the diagnosis of electric motor bearings, by the automatic classification of the various operating conditions the machine .The signals taken from the experimental test rig are processed by using various methods of signal processing. The calculated indicators were used to build the patterns vector, which is used for the following to train and test of the network. The GA are used to search(optimize) the structure and the various parameters of the network, which simplifies the neural network structure and makes the training process more efficient and giving the best performances of the network.