Study on Neural Networks in Parallel Computing Environments

The Sixth International Symposium on Neural Networks (ISNN 2009)

Soft-computing tools such as genetic algorithms and neural networks have found increased use in problems of engineering design. Applications have included the use of neural networks as response surface-like function approximations, and in identifying causality in numerical data. Both neural networks and genetic algorithms have also been used in the solution of generically difficult design optimization problems. A more recent application of these tools is in the field of computational intelligence. The present paper explores the use of a machine learning paradigm referred to as a classifier system, the central building block of which is a genetic algorithm, to provide a rational approach through which to construct neural network based global function approximations. The emphasis of the approach is to determine through computations, an optimal distribution of data which maximally improves the quality of the function approximation available from a back-propagation neural network. Numerical results are presented in support of the proposed scheme.

Fully sequential ranking-and-selection (R&S) procedures to find the best from a finite set of simulated alternatives are often designed to be implemented on a single processor. However, parallel computing environments, such as multi-core personal computers and many-core servers, are becoming ubiquitous and easily accessible for ordinary users. In this paper, we propose two types of fully sequential procedures that can be used in parallel computing environments. We call them vector-filling procedures and asymptotic parallel selection procedures, respectively. Extensive numerical experiments show that the proposed procedures can take advantage of multiple parallel processors and solve large-scale R&S problems.

Chapter 50 - Security Considerations in the Distributed Parallel Computation Environment

Backpropagation neural network has been applied successfully to solving uncertain problems in many fields. However, unsolved drawbacks still exist such as the problems of local minimum, slow convergence speed, and the determination of initial weights and the number of processing elements. In this paper, we introduce a single-layer orthogonal neural network (ONN) that is developed based on orthogonal functions. Since the processing elements are orthogonal to one another and there is no local minimum of the error function, the orthogonal neural network is able to avoid the above problems. Among the five existing orthogonal functions, Legendre polynomials and Chebyshev polynomials of the first kind have the properties of recursion and completeness. They are the most suitable to generate the neural network. Some typical examples are given to show their performance in function approximation. The results show that ONN has excellent convergence performance. Moreover, ONN is capable of approximating the mathematic model of backpropagation neural network. Therefore, it should be able to be applied to various applications that backpropagation neural network is suitable to solve. © 2001 John Wiley & Sons, Inc.

The article discusses the prerequisites and natural consequences of the control methods development in engineering systems: (1) a simple deviation and perturbation controller, (2) a fuzzy logic controller with a fuzzifier and a rule base, (3) a neural network controller for dynamically adjusting the coefficients of the corresponding links, (4) a discrete neural network controller with a neural approximator and controller. The experience gained by researchers and engineers since the first description of the principles of regulation in 1910 and the level of information technologies design, in particular the neural network method of machine learning and the enormous computing potential of computer devices, today can be integrated into a fundamentally new method of discrete neural network regulation.
 The review carried out in the article is aimed at identifying and demonstrating the significance of experimental and operational data, which must be properly structured and marked up at the stage of their collection and archiving. It is this approach that will allow us to quickly implement neural network controllers in engineering systems, since the most important stage for their creation is the process of learning and optimizing the architecture of neural networks.
 The principle of operation, advantages and disadvantages, the optimal stages of a neural network controller improvement based on two neural networks for the formation of a control strategy, taking into account the most probable state of the system at the next point in time, are given.

Weighted neural network and weighted least square estimators under censorship.

This paper presents a new single-layer neural network which is based on orthogonal functions. This neural network is developed to avoid the problems of traditional feedforward neural networks such as the determination of initial weights and the numbers of layers and processing elements. The desired output accuracy determines the required number of processing elements. Because weights are unique, the training of the neural network converges rapidly. An experiment in approximating typical continuous and discrete functions is given. The results show that the neural network has excellent performance in convergence time and approximation error.

Adaptive Sequential Posterior Simulators for Massively Parallel Computing Environments

A numerical method is developed for solving parabolic partial differential equations with integral boundary conditions. The method is moderately sixth-order accurate due to merging of sixth order finite difference scheme and fifth order Pade’s approximation. Simpson’s 1/3 rule is used to approximate integral conditions. The method does not involve the use of complex arithmetic and optimizes the results. It is observed that this numerical method can be easily coded on serial as well as parallel computers.

Application of the parallel diagonal dominant algorithm for the incompressible Navier-Stokes equations

Backgammon is a board game that has been studied considerably by computer scientists. Apart from standard backgammon, several yet unexplored variants of the game exist, which use the same board, number of checkers, and dice but may have different rules for moving the checkers, starting positions and movement direction. This paper studies two popular variants in Greece and neighboring countries, named Fevga and Plakoto. Using reinforcement learning and Neural Network function approximation we train agents that learn a game position evaluation function for these games. We show that the resulting agents significantly outperform the open-source program Tavli3D.

This article presents a review of various hardware and software parallel computing environments so far developed for the implementation of parallel algorithms to solve complex and computationally dominated optimization problems. These parallel environments are composed of parallel programming tools, performance evaluation tools, debuggers and optimization libraries. The environments can be classified based on the ease of implementation, hardware independence, ease to understand, and guarantee of performance.

In this paper, we propose an effective method of $$\hbox {PM}_{2.5}$$ prediction based on image contrast-sensitive features and weighted bagging based neural network (WBBNN). Three types of image contrast-sensitive features are first extracted from the images and fuzzified. Next, a weighted bagging strategy combining the ensemble fuzzy neural network (FNN) and ensemble radial basis function neural network (RBFNN) is established. The ensemble neural network (NN), regardless of FNN and RBFNN, is obtained by simply averaging the outputs of component neural networks. And these component neural networks are trained by the improved gradient descent algorithm and samples acquired by bootstrap sampling. Finally, the WBBNN is used to forecast $$\hbox {PM}_{2.5}$$ concentration by extracting three types of image contrast-sensitive features. Results of experiments demonstrate that our prediction method is more reliable, practical and efficient than FNN, RBFNN, the ensemble NNs, and state-of-the-art quality assessment method in terms of predicting the concentration of $$\hbox {PM}_{2.5}$$. More importantly, an improved gradient descent algorithm is developed to accelerate the convergence speed and ensure the prediction accuracy of WBBNN and the fuzzy features acquired by the feature fuzzification method can greatly improve the robustness and precision of WBBNN.

This paper describes four neural networks multilayer perceptron (MLP) network, Elman network, NARXSP network and radial basis function (RBF) network. Neural networks are applied for identification and control of DC servo motor and benchmark nonlinear system. Number of epochs required and time taken to train the controller are shown in the form of bar plots for four neural networks. Levenberg-Marquardt algorithm is used for training the controller using neural network toolbox in MATLAB. Each neural network controller is run ten times. Their performances are compared for each run in terms of number of epochs required and time taken to train each controller for tracking a reference trajectory.