The Self-Organizing Maps: Background, Theories, Extensions and Applications

This paper presents the development of several efficient learning algorithms for neural networks (ELEANNE). The ELEANNE 1 and ELEANNE 2 are two recursive least-squares learning algorithms, proposed for training single-layered neural networks with analog output. This paper also proposes a new optimization strategy for training single-layered neural networks, which provides the basis for the development of a variety of efficient learning algorithms. This optimization strategy is the source of the ELEANNE 3, a second-order learning algorithm for training single-layered neural networks with binary output. A simplified version of this algorithm, called ELEANNE 4, is also derived on the basis of some simplifying but reasonable assumptions. The two algorithms developed for single-layered neural networks provide the basis for the derivation of ELEANNE 5 and ELEANNE 6, which are proposed for training multilayered neural networks with binary output. The ELEANNE 7 is an efficient algorithm developed for training multilayered neural networks with either binary or analog output. >

A VLSI bio-inspired microsystem for real-time, on-site, robust image analysis and recognition has been developed. A corresponding novel artificial neural network (ANN) learning algorithm using new sigmoid-logarithmic transfer function is invented. Our results show the trained new ANN can recognize low fluorescence patterns better than the conventional sigmoidal ANN does. A differential logarithmic imaging chip is designed for calculating logarithm of relative intensities of fluorescence signals. The single-rail logarithmic circuit and a prototype ANN chip are designed, fabricated and characterized.

Artificial neural networks provide a neurocomputing approach for solving complex problems that might otherwise not have a tractable solution. Neural networks are usually trained using local gradient-based procedures. Such methods are frequently find sub-optimal solutions being trapped in local minima. And It will be difficult for the traditional neural network to solve such problems where the input/output data sets used to train a neural network may not be hundred percent precise but within certain range. A learning algorithm based on interval optimization is presented in this paper. The above disadvantages of the traditional learning algorithm can be settled by using this method.

At present, deep learning is becoming one of the most popular approach to creation of the artificial intelligences systems such as speech recognition, natural language processing, computer vision and so on. The paper presents a historical overview of deep learning in neural networks. The model of the artificial neural network is described as well as the learning algorithms for neural networks including the error backpropagation algorithm, which is used to train deep neural networks. The development of neural networks architectures is presented including neocognitron, autoencoders, convolutional neural networks, restricted Boltzmann machine, deep belief networks, long short-term memory, gated recurrent neural networks, and residual networks. Training deep neural networks with many hidden layers is impeded by the vanishing gradient problem. The paper describes the approaches to solve this problem that provide the ability to train neural networks with more than hundred layers. An overview of popular deep learning libraries is presented. Nowadays, for computer vision tasks convolutional neural networks are utilized, while for sequence processing, including natural language processing, recurrent networks are preferred solution, primarily long short-term memory networks and gated recurrent neural networks.

The Sixth International Symposium on Neural Networks (ISNN 2009)

A microsystem architecture for real-time, on-site, robust bioinformatic patterns recognition and analysis has been proposed. This system is compatible with on-chip DNA analysis means such as polymerase chain reaction (PCR) amplification. A corresponding novel artificial neural network (ANN) learning algorithm using new sigmoid-logarithmic transfer function based on error backpropagation (EBP) algorithm is invented. Our results show the trained new ANN can recognize low fluorescence patterns better than the conventional sigmoidal ANN does. A differential logarithmic imaging chip is designed for calculating logarithm of relative intensities of fluorescence signals. The single-rail logarithmic circuit and a prototype ANN chip are designed, fabricated and characterized

This article proposes a new method for interpreting computations performed by populations of spiking neurons. Neural firing is modeled as a rate-modulated random process for which the behavior of a neuron in response to external input can be completely described by its tuning function. I show that under certain conditions, cells with any desired tuning functions can be approximated using only spike coincidence detectors and linear operations on the spike output of existing cells. I show examples of adaptive algorithms based on only spike data that cause the underlying cell-tuning curves to converge according to standard supervised and unsupervised learning algorithms. Unsupervised learning based on principal components analysis leads to independent cell spike trains. These results suggest a duality relationship between the random discrete behavior of spiking cells and the deterministic smooth behavior of their tuning functions. Classical neural network approximation methods and learning algorithms based on continuous variables can thus be implemented within networks of spiking neurons without the need to make numerical estimates of the intermediate cell firing rates.

University admissions and business personnel offices use a limited number of resources to process an ever-increasing quantity of student and employment applications. Application systems are further constrained to identify and acquire, in a limited time period, those candidates who are most likely to accept an offer of enrolment or employment. Neural networks are a new methodology to this particular domain. Various neural network architectures and learning algorithms are analyzed comparatively to determine the applicability of supervised learning neural networks to the domain problem of personnel resource allocation and to identify optimal learning strategies in this domain. This paper focuses on multilayer perceptron backpropagation, radial basis function, counterpropagation, general regression, fuzzy ARTMAP, and linear vector quantization neural networks. Each neural network predicts the probability of enrolment and nonenrolment for individual student applicants. Backpropagation networks produced the best overall performance. Network performance results are measured by the reduction in counsellors student case load and corresponding increases in student enrolment. The backpropagation neural networks achieve a 56% reduction in counsellor case load.

Artificial neural networks (NNs) are highly parallel structures that consist of a large number of elementary nonlinear units (called neurons) fully interconnected. We present a hardware implementation of NNs combined with a DSP; both NNs and DSP combined together result in a powerful system used in control applications. NNs process most of the control tasks, while DSP performs signal preprocessing, signal conditioning and learning algorithms. In some cases the DSP performs the control of some discrete states of the plant by implementing finite state automata and/or verifying plant safety boundary operations. With this marriage, NN hardware can be very simple because several operations related with the NNs (learning algorithms, weight maintenance, etc.) can be performed by the DSP. The system implements intelligent control paradigms by mixing neuro fuzzy algorithms with finite state automata and/or digital control algorithms.

Spiking Self-organizing Maps for Classification Problem

There exist many learning algorithms for finding the synaptic weights of a layered neural network so as to minimize the output errors in response to the plural training data. The most well-known learning algorithms are the backpropagation procedures, in which there are two manners of minimizing the plural error functions in practice. One is minimizing the sum of all error functions totally, and the other is making a reduction of only one error function chosen temporarily and cyclically at each iteration. The former does not always keep the balance and the efficiency in reduction of the plural output errors, and the latter does not guarantee the convergence from the standpoint of numerical analysis.

The use of renewable energy sources in the production of electricity has become inevitable in order to reduce the greenhouse gases left in the atmosphere that cause the Earth to warm up. Although countries on a national basis have implemented a number of policies to support electricity generated from renewable energy sources, investments to produce electricity without a license on a local basis are not desirable. According to the climatic conditions of the power plant of 1 MW installed founded in Konya and power plant production data are monitored. Machine learning is a sub-branch of artificial intelligence that deals with the design and development of algorithms that allow computers to develop their behavior based on experimental data. In this study, Naive Bayes, Decision Tree, CN2 Rule Induction, Random Forest, Support Vector Machine, k-Nearest Neighbor, Artificial Neural Network, Logistic Regression and AdaBoost machine learning algorithms are used for prediction and classification. Generally, energy investors are curious about the return on their investment. It is very important for energy providers to predict how much electricity will be generated from existing solar power plants and accordingly determine the measures they will take to meet the electricity demand in the future. ROC analyzes were performed for machine learning models and performance evaluation was performed. In this study, the best performance estimation value obtained from the solar power plant depending on the weather conditions was obtained with 92.24% accuracy.

A problem of an adaptive neural network system synthesizes in a case of a nonlinear actuation channel's singularities is considered. The singularities can cause a loss of stability if the back propagation algorithm is used without modification. Known approaches of solving the problem are considered and a new modification of adaptive learning algorithm is proposed that can be applied to a wider class of processes.

This paper describes the time variant changes in satellite images using Self Organizing Feature Map (SOFM) technique associated with Artificial Neural Network. In this paper, we take a satellite image and find the time variant changes using above technique with the help of MATLAB. This paper reviews remotely sensed data analysis with neural networks. First, we present an overview of the main concepts underlying Artificial Neural Networks (ANNs), including the main architectures and learning algorithms. Then, the main tasks that involve ANNs in remote sensing are described. We first make a brief introduction to models of networks, for then describing in general terms Artificial Neural Networks (ANNs). As an application, we explain the back propagation algorithm, since it is widely used and many other algorithms are derived from it. There are two techniques that are used for classification in pattern recognition such as Supervised Classification and Unsupervised Classification. In supervised learning technique the network knows about the target and it has to change accordingly to get the desired output corresponding to the presented input sample data. Most of the previous work has already been done on supervised classification. In this study we are going to present the classification of satellite images using unsupervised classification method of ANN.

There is a growing need for a method to analyze massive gene expression data obtained from DNA microarrays. We introduce a method of combining a self-organizing map (SOM) and a k-means clustering for analyzing and categorizing the gene expression data. The SOM is an unsupervised neural network learning algorithm and forms a mapping the high-dimensional data to two-dimensional space. However, it is difficult to find clustering boundaries from results of the SOM. On the other hand, the k-means clustering can partition the data into the clusters under the assumption of the known number of clusters. In order to understand easily the results of SOM, we apply the k-means clustering to finding the clustering boundaries from results of SOM. We have applied the proposed method to the published data of yeast gene expression and show that the method is effective for categorizing the data.