Theoretical analysis of the batch variant of the self-organizing feature map algorithm for 1-d networks mapping a continuous 1-d input space

This paper describes two variants of the Kohonen's self-organizing feature map (SOFM) algorithm. Both variants update the weights only after presentation of a group of input vectors. In contrast, in the original algorithm the weights are updated after presentation of every input vector. The main advantage of these variants is to make available a finer grain of parallelism, for implementation on machines with a very large number of processors, without compromising the desired properties of the algorithm. In this work it is proved that, for one-dimensional (1-D) maps and 1-D continuous input and weight spaces, the strictly increasing or decreasing weight configuration forms an absorbing class in both variants, exactly as in the original algorithm. Ordering of the maps and convergence to asymptotic values are also proved, again confirming the theoretical results obtained for the original algorithm. Simulations of a real-world application using two-dimensional (2-D) maps on 12-D speech data are presented to back up the theoretical results and show that the performance of one of the variants is in all respects almost as good as the original algorithm. Finally, the practical utility of the finer parallelism made available is confirmed by the description of a massively parallel hardware system that makes effective use of the best variant.

Proposes a new scheme for designing a nearest-prototype classifier. The system starts with the minimum number of prototypes, equal to the number of classes. Kohonen's self-organizing feature map (SOFM) algorithm is used to obtain this initial set of prototypes. Then, on the basis of the classification performance, new prototypes are generated dynamically, similar prototypes are merged, and prototypes with less significance are deleted, leading to better performance. If prototypes are deleted or new prototypes appear, then they are retrained using Kohonen's SOFM algorithm with the winner-only update scheme. This adaptation continues until the system satisfies a termination condition. The classifier has been tested with several well-known data sets. The results obtained are quite satisfactory.

An unsupervised competitive learning rule, called the vectorial boundary adaptation rule (VBAR), is introduced for topographic map formation. Since VBAR is aimed at producing an equiprobable quantization of the input space, it yields a nonparametric model of the input probability density function. Furthermore, since equiprobable quantization is equivalent to unconditional entropy maximization, we argue that this is a plausible strategy for maximizing mutual information (Shannon information rate) in the case of "online" learning. We use mutual information as a tool for comparing the performance of our rule with Kohonen's self-organizing (feature) map algorithm.

This paper presents a neuro-fuzzy system by using the Kohonen's self-organizing feature map algorithm, not only for its vector quantization feature, but also for its topological property. This property prevents the proposed neuro-fuzzy system from suffering from a drawback like any of the conventional clustering-algorithm-based fuzzy systems, i.e. the optimal number of clusters or some kind of similarity threshold must be predetermined. Associated with the self-organizing feature-map-based fuzzy system is a hybrid learning algorithm, which is for initial parameters setting and fine-tuning the parameters of the system.

This paper presents an iterative fuzzy vector quantization approach used in codebook design and its neural net algorithm, the fuzzy self-organizing feature map (FSOFM) algorithm, which is the development of the self-organizing feature map (SOFM) algorithm and the fuzzy vector quantization (FVQ) algorithms. The FVQ algorithm allows that each training vector is assigned to multiple codewords in the early stages of the codebook design. ALthough, the FVQ algorithm reduces the dependence of the resulting codebook on the initial codebook, the codewords are calculated in batch mode. The iterative fuzzy vector quantization approach is based on a gradient decent approach, and the concept of fuzzy is introduced into it. The FSOFM algorithm considers the winning output node and its neighborhood as a fuzzy set of the input node. As a result, the feature vector of the output node in the fuzzy set of the input sample can be updated by the membership function and the training vector just completing once iteration. In this paper, the LBG, FVQ, SOFM and FSOFM algorithms are used in image compression based on vector quantization. This paper evaluates the computing efficiency of these algorithms and compares the quality of the resulting codebooks.© (1997) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

The magnification exponents μ occurring in adaptive map formation algorithms like Kohonen's self-organizing feature map deviate for the information theoretically optimal value μ = 1 as well as from the values that optimize, e.g., the mean square distortion error (μ = 1/3 for one-dimensional maps). At the same time, models for categorical perception such as the "perceptual magnet" effect, which are based on topographic maps, require negative magnification exponents μ < 0. We present an extension of the self-organizing feature map algorithm, which utilizes adaptive local learning step sizes to actually control the magnification properties of the map. By change of a single parameter, maps with optimal information transfer, with various minimal reconstruction errors, or with an inverted magnification can be generated. Analytic results on this new algorithm are complemented by numerical simulations.

We applied Kohonen's self-organizing feature maps (SOM) into the problem of selecting the buttons. 3D data spaces are represented by two maps of 2D SOM which are formed on the planes that are projected from 3D objects. Assuming the ability of SOM to approximate the shape of input-data spaces, we select the best matching color-series when reference-data point exists in the SOM. Using this method, very good recognition (96%) was obtained. This method is better than the neural network with backpropagation learning on the points that adjustments of parameters are easy and paradoxical data-relations are permitted.

The paper presents the design of municipal creditworthiness parameters. Further, a model is designed based on Learning Vector Quantization neural networks for municipal creditworthiness classification. The model is composed of Kohonen's Self-organizing Feature Maps (unsupervised learning) whose outputs represent the input of the Learning Vector Quantization neural networks (supervised learning).

To reduce the severe performance degradation of the traditional antenna selection schemes in correlated channels, this paper proposes a new antenna selection scheme via the self-organizing feature map (SOFM) algorithm which exploits spatial local related statistical information in fully correlated channels to improve the spatial adaptability. SOFM algorithm classifies N available antennas into L classes: antennas in each class are strongly correlated but antennas between different classes are weak correlated or uncorrelated. Thus, one antenna in each class offers the possibility of almost-optimum selection and increases the rank of the correlated fading channel matrix. These increments, directly, contribute to the Effective Degree Of Freedom (EDOF) of the channel and hence the performance. Through simulations, it was demonstrated that the proposed scheme provides excellent performance in correlated MIMO channels.

This paper addresses the problem of visual assessment of clustering tendency in p-dimensional data using two extensions of Kohonen's self-organizing feature map (SOFM). We show that SOFM cell displays generally do not produce visual evidence that leads to good guesses about cluster substructure or data density even for 2-dimensional data. The two proposed extensions of SOFM improve the quality of displays and enable us to make better guesses about the existence of substructure in data.

An investigation of local energy surface detection integrated with neural network techniques for image segmentation is presented, as applied in the feature extraction of chromosomes from image datasets obtained using an experimental confocal microscope. Use of the confocal microscope enables biologists to observe dividing cells (living or preserved) within a three-dimensional (3-D) volume, that can be visualised from multiple aspects, allowing for increased structural insight. The Nomarski differential interference contrast mode used for imaging translucent specimens, such as chromosomes, produces images not suitable for volume rendering. Segmentation of the chromosomes from this data is, thus, necessary. A neural network based on competitive learning, known as Kohonen's self-organizing feature map (SOFM) was used to perform segmentation, using a collection of statistics or features defining the image. Our past investigation showed that standard features such as the localized mean and variance of pixel intensities provided reasonable extraction of objects such as mitotic chromosomes, but surface detail was only moderately resolved. In this current work, a biologically inspired feature known as local energy is investigated as an alternative image statistic based on phase congruency in the image. This, along with different combinations of other image statistics, is applied in a SOFM, producing 3-D images exhibiting vast improvement in the level of detail and clearly isolating the chromosomes from the background. Index Terms-DIC, differential interference contrast, feature extraction, feature space, image segmentation, local energy, Morlet wavelet, phase congruency, self organizing feature map, SOFM.

Research on artificial neural networks (ANNs) has been carried out for more than five decades. A renewed interest appeared in the 80's with the finding of powerful models like J. Hopfield's recurrent networks, T. Kohonen's self-organizing feature maps, and the back-propagation rule. At that time, there was no platform that was at the same time versatile enough for any ANN model to be implemented and fast enough to solve large problems. Super-computers were the sole exception to this rule, but were prohibitively expensive for most applications. However, both research scientists and application engineers clearly identified the need for such a computing power. This triggered many projects in the field. In parallel, research on multi-processor systems started during the 60's. Systolic arrays have been proposed in 1979 as a means to fully exploit the possibilities of VLSI. Two previous theses, by F. Blayo and C. Lehmann, have studied the use of bi-dimensional systolic arrays for neural computation. At first, the presented system, called GENES, has been designed for the Hopfield model. Extensions to other ANNs have also been proposed. The goal of the present thesis is to study, design, build, and analyze an efficient accelerator for neural computation. In a first step, the GENES architecture has been extended towards generality and efficiency. This includes a thorough analysis of ANN models, of other neural computers, and of previous GENES implementations. The result of this work is the GENES IV integrated circuit, whose architecture has been co-designed by P. Ienne and the author. The main part of this thesis discusses the architecture, the design and the analysis of the MANTRA I machine, a neural computer based on a GENES IV array with up to 40 × 40 processing elements (PEs). The delta rule (and hence the Perceptron and Adaline rules), the back-propagation rule, the Hopfield model, and the Kohonen model can be implemented on this system. Although not a generic system, such a machine may be regarded as a multi-model neural computer. A prototype has been running for a year and is used daily by software designers. Several novel features distinguish the MANTRA I machine from other neural systems. First, it belongs to the few existing neural computers, contrary to the majority of implementations, which are specific to an application or to an algorithm. The machine does not hard-wire any algorithm, but provides the necessary primitives to implement the target models. This is a key feature for research, since several algorithms or versions of an algorithm can be tested on a problem. It is an important aspect for applications as well, because different ANN models are often cascaded to solve a problem. The GENES IV array — that is, the computing core of the MANTRA I machine — features synapse-level parallelism (i.e., one real or virtual PE is allocated per synapse or neural connection), while most other systems exploit only neuron-level parallelism (i.e., one PE per neuron). Hence, this system aims at a much finer parallelism grain and is well suited for massively parallel architectures. The problem size that can be computed by a neural accelerator should not be limited by the hardware (except for memory size). Therefore, it is essential to support time-sharing of PEs. On the MANTRA I machine, this is achieved by the concept of virtual arrays. Matrices are divided into sub-matrices that can be mapped onto the physical array, which is then time-shared among them. An efficient mechanism has been implemented to swap sub-matrices in background, while some other computation is performed. Since systolic arrays are pipelined systems, it is important to avoid emptying and re-filling them too often, in order to keep the hardware utilization rate high. Therefore, a systolic instruction flow has been implemented, so that each instruction follows the data for which it has been issued. Like any SIMD system, the MANTRA I machine is composed of a parallel or SIMD module and a control module. The SIMD module consists of the GENES IV array and a set of dedicated units designed for the computation that scales with the number of neurons and would poorly fit on a bi-dimensional array. A complex system of FIFOs and memories sustains the required input/output streams for the systolic array. The control module is a complete SISD system. Its tasks are (1) to control the SIMD module by dispatching instructions, (2) to manage data input and output, (3) to communicate with the external world, and (4) to perform data pre- and post-processing. The SIMD instructions are of the very long instruction word (VLIW) type. Synchronization between the two modules is achieved by an instruction FIFO. The performance of the MANTRA I machine has been analyzed using the delta rule. Measurements show that the sustained performance is very close to its peak value, as long as the problem fits in the memory banks connected to the GENES IV array. Experiments have also been run to investigate the impact of the constraints imposed by the hardware on the convergence of algorithms. Finally, the use of systolic arrays as neural accelerators is discussed in the light of the experience acquired with the GENES IV array and the MANTRA I machine. The weaknesses of the machine are analyzed, and several solutions are proposed to avoid them in a future design. A general discussion of the future of neural computers concludes this thesis.

It is often reported in the technique literature that the success of the self-organizing feature map formation is critically dependent on the initial weights and the selection of main parameters (i.e. the learning-rate parameter and the neighborhood set) of the algorithm. They usually have to be counteracted by the trial-and-error method; therefore, often time consuming retraining procedures have to precede before a neighborhood preserving feature amp is obtained. In this paper, we propose an efficient initialization scheme to construct an initial map. We then use the self-organizing feature map algorithm to make small subsequent adjustments so as to improve the accuracy of the initial map. Several data sets are tested to illustrate the performance of the proposed method.

<p>The shapes and firing rates of motor unit action potentials (MUAPs) in an electromyographic (EMG) signal provide an important source of information for the diagnosis of neuromuscular disorders. In order to extract this information from EMG signals recorded at force levels up to 20% of maximum voluntary contraction (MVC) it is required: (i) To identify the MUAPs composing the EMG signal, (ii) To classify MUAPs with similar shape and (iii) To decompose the superimposed MUAP waveforms into their constituent MUAPs. For the classification of MUAPs two different pattern recognition techniques are presented (i) An artificial neural network (ANN) technique based on unsupervised learning using the self-organizing feature maps (SOFM) algorithm and learning vector quantization (LVQ) and (ii) A statistical pattern recognition technique based on the euclidian distance. The success rate on real data for the ANN technique is about 96% and for the statistical one about 94%. For the decomposition of the superimposed waveforms the following technique is used: (i) Cross-correlation of each of the unique MUAP waveforms, obtained by the classification process with the superimposed waveforms in order to find the best matching point and (ii) A combination of euclidian distance and area measures in order to classify the components of the decomposed waveform. The success rate for the decomposition procedure is about 90%.</p>

A multilayer neural nerwork model for the perception of rotational motion has been developed using Reichardt's motion detector array of correlation type, Kohonen's self-organized feature map and Schuster-Wagner's oscillating neural network. It is shown that the unsupervised learning could make the neurons on the second layer of the network tend to be self-organized in a form resembling columnar organization of selective directions in area MT of the primate's visual cortex. The output layer can interpret rotation information and give the directions and velocities of rotational motion. The computer simulation results are in agreement with some psychophysical observations of rotational perception. It is demonstrated that the temporal correlation between the oscillating neurons would be powerful for solving the "binding problem" of shear components of rotational motion.