Interpretation of perceptron weights as constructed time series for EEG classification

Chapter 5 - Training multilayer neural networks

This paper presents the development of several efficient learning algorithms for neural networks (ELEANNE). The ELEANNE 1 and ELEANNE 2 are two recursive least-squares learning algorithms, proposed for training single-layered neural networks with analog output. This paper also proposes a new optimization strategy for training single-layered neural networks, which provides the basis for the development of a variety of efficient learning algorithms. This optimization strategy is the source of the ELEANNE 3, a second-order learning algorithm for training single-layered neural networks with binary output. A simplified version of this algorithm, called ELEANNE 4, is also derived on the basis of some simplifying but reasonable assumptions. The two algorithms developed for single-layered neural networks provide the basis for the derivation of ELEANNE 5 and ELEANNE 6, which are proposed for training multilayered neural networks with binary output. The ELEANNE 7 is an efficient algorithm developed for training multilayered neural networks with either binary or analog output. >

Phishing is an attempt by an individual or a group of person to steal personal information such as password, banking account and credit card information, etc. Most of these phishing web pages look similar to the real web pages in terms of website interface and uniform resource locator (URL) address. Many techniques have been proposed to detect phishing websites, such as Blacklist-based technique, Heuristic-based technique, etc. However, these techniques are still inefficient. While Blacklist-based techniques cannot detect the phishing sites that are not in the blacklist database, the weights of the heuristic in the heuristic-based approach significantly depend on the statistics from the training dataset. Therefore, a new phishing detection approach in which either the new phishing websites can be detected or the weights of the heuristic are derived objectively is necessary. In this paper, an efficient approach for detecting phishing websites based on the single-layer neural network is proposed. Specifically, the proposed technique calculates the value of heuristics objectively. Then, the weights of heuristic are generated by a single-layer neural network. The proposed technique is evaluated with a dataset of 11,660 phishing sites and 10,000 legitimate sites. The results show that the technique can detect over 98% phishing sites.

This paper proposes and investigates an on-chip diffractive optical neural network based on binary metasurfaces. The genetic algorithm and finite-difference time-domain method are used to optimize the binary metasurfaces to achieve relatively compact area size and excellent performance. To prove the effectiveness of our proposal, a single-layer diffractive optical neural network based on binary metasurfaces is designed to execute a classification task on the Iris dataset. The area size of the designed single-layer diffractive optical neural network is only 16.5 µm × 23 µm. In the simulation, a validation accuracy of 90.0% is attained. The designed single-layer diffractive optical neural network was fabricated on a 220-nm silicon-on-insulator platform as a proof of concept. Measurement results show that the validation accuracy of the fabricated single-layer diffractive optical neural network is 78.3%.

Machine Learning is an area concerned with the automation of the process of knowledge acquisition. Neural networks generally represent their knowledge at the lower level, while knowledge based systems use higher level knowledge representations. The method we propose here, provides a technique which automatically allows us to extract production rules from the lower level representation used by a single-layered neural networks trained by Hebb's rule. Even though a single-layered neural network can not model complex, nonlinear domains, their strength in dealing with noise has enabled us to produce correct rules in a noisy domain.

In this paper, a hybrid cuckoo search technique is combined with a single-layer neural network (BHCS-ANN) to approximate the solution to a differential equation describing the curvature shape of the cornea of the human eye. The proposed problem is transformed into an optimization problem such that the L2–error remains minimal. A single hidden layer is chosen to reduce the sink of the local minimum values. The weights in the neural network are trained with a hybrid cuckoo search algorithm to refine them so that we obtain a better approximate solution for the given problem. To show the efficacy of our method, we considered six different corneal models. For validation, the solution with Adam’s method is taken as a reference solution. The results are presented in the form of figures and tables. The obtained results are compared with the fractional order Darwinian particle swarm optimization (FO-DPSO). We determined that results obtained with BHCS-ANN outperformed the ones acquired with other numerical routines. Our findings suggest that BHCS-ANN is a better methodology for solving real-world problems.

The mapping of land cover and land use is a key application of remotely sensed data. Studies have suggested the outputs of statistical models that estimate the posterior probability of class membership can be interpreted as subpixel area proportions. This paper examines the correlation between posterior probability of class membership, estimated using neural network and nearest neighbour models, and area proportion. In addition, the paper describes several models, again based on neural networks and nearest neighbour algorithms, that have been developed to estimate the land cover area proportions explicitly. Both types of model were applied to a Landsat TM data set. The results demonstrated that better estimates of the true land cover area were obtained using models that predicted the area proportion directly than were obtained using models that predicted the posterior probability of class membership. Further, it was found that a linear model (single-layer neural network) and a nearest neighbour smoothing model produced higher correlation and lower errors than the other models investigated.

Forecasting Foreign Exchange Rate Movements with k-Nearest-Neighbour, Ridge Regression and Feed-Forward Neural Networks

The scattering effect of turbid media can lead to optical wavefront distortion. Focusing light through turbid media can be achieved using wavefront shaping techniques. Intelligent optimization algorithms and neural network algorithms are two powerful types of algorithms in the field of wavefront shaping but have their advantages and disadvantages. In this paper, we propose a new hybrid algorithm that combines the particle swarm optimization algorithm (PSO) and single-layer neural network (SLNN) to achieve the complementary advantages of both. A small number of training sets are used to train the SLNN to obtain preliminary focusing results, after which the PSO continues to optimize to the global optimum. The hybrid algorithm achieves faster convergence and higher enhancement than the PSO, while reducing the size of training samples required for SLNN training. SLNN trained with 1700 training sets can speed up the convergence of the PSO by about 50% and boost the final enhancement by about 24%. This hybrid algorithm will be of great significance in fields such as biomedicine and particle manipulation.

This paper presents a complex-valued neuron (CVN) model for real-valued classification problems incorporating a new activation function. The activation function maps complex-valued net-inputs (sum of weighted inputs) of a neuron into bounded real-values, and its role is to divide the net-input space into different regions for different classes. A gradient-descent learning rule has been derived to train the CVN. Such a CVN is able to solve all possible two-input Boolean functions. For further investigation, single layered complex-valued neural networks (Le. without hidden units) are applied on the real-world multi-class classification problems. The results are comparable to the conventional multilayer real-valued neural networks. It is also shown that the performance can be improved further by using their ensembles. Negative correlation learning (NCL) algorithm has been used to create the ensembles. Since NCL is a gradient-descent based algorithm, the proposed activation function is well suited for it.

This paper presents a new single-layer neural network which is based on orthogonal functions. This neural network is developed to avoid the problems of traditional feedforward neural networks such as the determination of initial weights and the numbers of layers and processing elements. The desired output accuracy determines the required number of processing elements. Because weights are unique, the training of the neural network converges rapidly. An experiment in approximating typical continuous and discrete functions is given. The results show that the neural network has excellent performance in convergence time and approximation error.

Adaptive RTRL based neurocontroller for damping subsynchronous oscillations using TCSC

Drowsiness can impair reaction time, increasing the risk of severe accidents. Many current studies concentrate on a single symptom of drowsiness, which can lead to false alerts. This paper presents a new method for detecting drowsiness in real time. The proposed approach utilizes four deep learning architectures based on convolutional neural networks: AlexNet for extracting environmental features, ResNet50V2 for recognizing hand gestures, VGG-FaceNet for facial feature extraction, and FlowImageNet for analyzing behavioral features. To maximize the benefits of the aforementioned methods, we suggest using a single-layer neural network. Since drowsiness is a dynamic phenomenon, capturing its evolving features requires a dynamic neural network with adaptive delays, specifically an Adaptive Time Delay Neural Network (ATDNN) with adjustable weights. Our implementation of this neuro-dynamic approach on the NTHUDDD and our custom datasets demonstrates that it achieves greater accuracy (99.1% and 98.6%, respectively) compared to existing methods in the literature.

Recent advances in deep learning have bolstered our ability to forecast the evolution of dynamical systems, but common neural networks do not adhere to physical laws, critical information that could lead to sounder state predictions. This contribution addresses this concern by proposing a neural network to polynomial (NN-Poly) approximation, a method that furnishes algorithmic guarantees of adhering to physics while retaining state prediction accuracy. To achieve these goals, this article shows how to represent a trained fully connected perceptron, convolution, and recurrent neural networks of various activation functions as Taylor polynomials of arbitrary order. This solution is not only analytic in nature but also least squares optimal. The NN-Poly system identification or state prediction method is evaluated against a single-layer neural network and a polynomial trained on data generated by dynamic systems. Across our test cases, the proposed method maintains minimal root mean-squared state error, requires few parameters to form, and enables model structure for verification and safety. Future work will incorporate safety constraints into state predictions, with this new model structure and test high-dimensional dynamical system data.

Nonlinear subspace clustering by functional link neural networks