An orthogonal neural network for function approximation

Backpropagation neural network has been applied successfully to solving uncertain problems in many fields. However, unsolved drawbacks still exist such as the problems of local minimum, slow convergence speed, and the determination of initial weights and the number of processing elements. In this paper, we introduce a single-layer orthogonal neural network (ONN) that is developed based on orthogonal functions. Since the processing elements are orthogonal to one another and there is no local minimum of the error function, the orthogonal neural network is able to avoid the above problems. Among the five existing orthogonal functions, Legendre polynomials and Chebyshev polynomials of the first kind have the properties of recursion and completeness. They are the most suitable to generate the neural network. Some typical examples are given to show their performance in function approximation. The results show that ONN has excellent convergence performance. Moreover, ONN is capable of approximating the mathematic model of backpropagation neural network. Therefore, it should be able to be applied to various applications that backpropagation neural network is suitable to solve. © 2001 John Wiley & Sons, Inc.

Tan spot of wheat, caused by Pyrenophora tritici-repentis, provided a model system for testing disease forecasts based on an artificial neural network. Infection periods for P. tritici-repentis on susceptible wheat cultivars were identified from a bioassay system that correlated tan spot incidence with crop growth stage and 24-h summaries of environmental data, including temperature, relative humidity, wind speed, wind direction, solar radiation, precipitation, and flat-plate resistance-type wetness sensors. The resulting data set consisted of 97 discrete periods, of which 32 were reserved for validation analysis. Neural networks with zero to nine processing elements were evaluated 20 times each to identify the model that most accurately predicted an infection event. The 200 models averaged 74 to 77% accuracy, depending on the number of processing elements and random initialization of coefficients. The most accurate model had five processing elements and correctly predicted 87% of the infection periods in the validation set. In comparison, stepwise logistic regression correctly predicted 69% of the validation cases, and multivariate discriminant analysis distinguished 50% of the validation cases. When wetness-sensor inputs were withheld from the models, both the neural network and logistic regression models declined 6% in prediction accuracy. Thus, neural networks were more accurate than statistical procedures, both with and without wetness-sensor inputs. These results demonstrate the applicability of neural networks to plant disease forecasting.

Chapter 5 - Training multilayer neural networks

This article discusses the application of orthogonal neural networks to detect collisions between multiple robot manipulators that work in an overlapped space. By applying an expansion/shrinkage algorithm, the problem of collision detection between arms is transformed into that among cylinders (or rectangular solids) and line segments. This mapping simplifies the collision detection problem and thus neural networks can be applied to solve it. The property of parallel processing enables neural networks to detect collisions rapidly. A single‐layer orthogonal neural network is developed to avert the problems of conventional multilayer feedforward neural networks such as initial weights and the number of layers and processing elements. This orthogonal neural network can approximate various functions and is used to calculate forward solution and to detect collisions. An efficient neural network system for collision detection is also developed. © 1995 John Wiley & Sons, Inc.

This paper presents the development of several efficient learning algorithms for neural networks (ELEANNE). The ELEANNE 1 and ELEANNE 2 are two recursive least-squares learning algorithms, proposed for training single-layered neural networks with analog output. This paper also proposes a new optimization strategy for training single-layered neural networks, which provides the basis for the development of a variety of efficient learning algorithms. This optimization strategy is the source of the ELEANNE 3, a second-order learning algorithm for training single-layered neural networks with binary output. A simplified version of this algorithm, called ELEANNE 4, is also derived on the basis of some simplifying but reasonable assumptions. The two algorithms developed for single-layered neural networks provide the basis for the derivation of ELEANNE 5 and ELEANNE 6, which are proposed for training multilayered neural networks with binary output. The ELEANNE 7 is an efficient algorithm developed for training multilayered neural networks with either binary or analog output. >

In recent years, the research of Multi-Agent Systems (MAS) has become a hot spot in complex systems research. It is widely used in multi-manipulator cooperative equipment, parallel computing, and network resource allocation. The planning of a multi-agent system is more and more critical. This paper takes RoboCup, a classic MAS, as an example. First, it is modeled mathematically through the Markov decision-making process, and a new ball-passing strategy solution based on a deep neural network is proposed. In addition, this paper also tries to use the traditional single-layer neural network to solve the same problem and compares the accuracy difference and model performance of the two types of neural networks. Finally, the possibility of applying a deep learning network in MAS is analyzed. The results show that the deep neural network has high effectiveness in solving the problem of the passing strategy of RoboCup. The ball control rate in the simulation environment is far more than the traditional algorithm, and the model trained by the deep learning network is significantly higher than the traditional neural network in accuracy, efficiency, and other aspects. Applying deep learning networks in other MAS is also likely to have a good effect. In the future, deep learning networks will be an essential planning algorithm.

This paper proposes and investigates an on-chip diffractive optical neural network based on binary metasurfaces. The genetic algorithm and finite-difference time-domain method are used to optimize the binary metasurfaces to achieve relatively compact area size and excellent performance. To prove the effectiveness of our proposal, a single-layer diffractive optical neural network based on binary metasurfaces is designed to execute a classification task on the Iris dataset. The area size of the designed single-layer diffractive optical neural network is only 16.5 µm × 23 µm. In the simulation, a validation accuracy of 90.0% is attained. The designed single-layer diffractive optical neural network was fabricated on a 220-nm silicon-on-insulator platform as a proof of concept. Measurement results show that the validation accuracy of the fabricated single-layer diffractive optical neural network is 78.3%.

Appropriate definition of neural network architecture prior to data analysis is crucial for successful data mining. This is particularly important when the underlying model of the data is unknown. The proposed algorithm is intended to develop automatically an appropriate neural network (including the number of layers, the number of processing elements per layer, and types of each processing element) needed to solve the given problem. Genetic programming (GP) is used to develop the neural network’s structure and the resilient-back-propagation (RPROP) will be used to train the neural network.

The soft computing algorithms are being nowadays used for various multi input multi output complicated non linear control applications. This paper presented the development and implementation of back propagation of multilayer perceptron architecture developed in FPGA using VHDL. The usage of the FPGA (Field Programmable Gate Array) for neural network implementation provides flexibility in programmable systems. For the neural network based instrument prototype in real time application. The conventional specific VLSI neural chip design suffers the limitation in time and cost. With low precision artificial neural network design, FPGA have higher speed and smaller size for real time application than the VLSI design. The challenges are finding an architecture that minimizes the hardware cost, maximizing the performance, accuracy. The goal of this work is to realize the hardware implementation of neural network using FPGA. Digital system architecture is presented using Very High Speed Integrated Circuits Hardware Description Language (VHDL)and is implemented in FPGA chip. MATLAB ANN programming and tools are used for training the ANN. The trained weights are stored in different RAM, and is implemented in FPGA. The design was tested on a FPGA demo board. Keywords- Backpropagation, field programmable gate array (FPGA) hardware implementation, multilayer perceptron, pressure sensor, Xilinx FPGA.

The scattering effect of turbid media can lead to optical wavefront distortion. Focusing light through turbid media can be achieved using wavefront shaping techniques. Intelligent optimization algorithms and neural network algorithms are two powerful types of algorithms in the field of wavefront shaping but have their advantages and disadvantages. In this paper, we propose a new hybrid algorithm that combines the particle swarm optimization algorithm (PSO) and single-layer neural network (SLNN) to achieve the complementary advantages of both. A small number of training sets are used to train the SLNN to obtain preliminary focusing results, after which the PSO continues to optimize to the global optimum. The hybrid algorithm achieves faster convergence and higher enhancement than the PSO, while reducing the size of training samples required for SLNN training. SLNN trained with 1700 training sets can speed up the convergence of the PSO by about 50% and boost the final enhancement by about 24%. This hybrid algorithm will be of great significance in fields such as biomedicine and particle manipulation.

Interpretation of perceptron weights as constructed time series for EEG classification

Machine Learning is an area concerned with the automation of the process of knowledge acquisition. Neural networks generally represent their knowledge at the lower level, while knowledge based systems use higher level knowledge representations. The method we propose here, provides a technique which automatically allows us to extract production rules from the lower level representation used by a single-layered neural networks trained by Hebb's rule. Even though a single-layered neural network can not model complex, nonlinear domains, their strength in dealing with noise has enabled us to produce correct rules in a noisy domain.

Drowsiness can impair reaction time, increasing the risk of severe accidents. Many current studies concentrate on a single symptom of drowsiness, which can lead to false alerts. This paper presents a new method for detecting drowsiness in real time. The proposed approach utilizes four deep learning architectures based on convolutional neural networks: AlexNet for extracting environmental features, ResNet50V2 for recognizing hand gestures, VGG-FaceNet for facial feature extraction, and FlowImageNet for analyzing behavioral features. To maximize the benefits of the aforementioned methods, we suggest using a single-layer neural network. Since drowsiness is a dynamic phenomenon, capturing its evolving features requires a dynamic neural network with adaptive delays, specifically an Adaptive Time Delay Neural Network (ATDNN) with adjustable weights. Our implementation of this neuro-dynamic approach on the NTHUDDD and our custom datasets demonstrates that it achieves greater accuracy (99.1% and 98.6%, respectively) compared to existing methods in the literature.

Recent advances in deep learning have bolstered our ability to forecast the evolution of dynamical systems, but common neural networks do not adhere to physical laws, critical information that could lead to sounder state predictions. This contribution addresses this concern by proposing a neural network to polynomial (NN-Poly) approximation, a method that furnishes algorithmic guarantees of adhering to physics while retaining state prediction accuracy. To achieve these goals, this article shows how to represent a trained fully connected perceptron, convolution, and recurrent neural networks of various activation functions as Taylor polynomials of arbitrary order. This solution is not only analytic in nature but also least squares optimal. The NN-Poly system identification or state prediction method is evaluated against a single-layer neural network and a polynomial trained on data generated by dynamic systems. Across our test cases, the proposed method maintains minimal root mean-squared state error, requires few parameters to form, and enables model structure for verification and safety. Future work will incorporate safety constraints into state predictions, with this new model structure and test high-dimensional dynamical system data.

This paper presents a complex-valued neuron (CVN) model for real-valued classification problems incorporating a new activation function. The activation function maps complex-valued net-inputs (sum of weighted inputs) of a neuron into bounded real-values, and its role is to divide the net-input space into different regions for different classes. A gradient-descent learning rule has been derived to train the CVN. Such a CVN is able to solve all possible two-input Boolean functions. For further investigation, single layered complex-valued neural networks (Le. without hidden units) are applied on the real-world multi-class classification problems. The results are comparable to the conventional multilayer real-valued neural networks. It is also shown that the performance can be improved further by using their ensembles. Negative correlation learning (NCL) algorithm has been used to create the ensembles. Since NCL is a gradient-descent based algorithm, the proposed activation function is well suited for it.