Speech recognition based on Kohonen self-organizing feature maps and hybrid connectionist systems

We applied Kohonen's self-organizing feature maps (SOM) into the problem of selecting the buttons. 3D data spaces are represented by two maps of 2D SOM which are formed on the planes that are projected from 3D objects. Assuming the ability of SOM to approximate the shape of input-data spaces, we select the best matching color-series when reference-data point exists in the SOM. Using this method, very good recognition (96%) was obtained. This method is better than the neural network with backpropagation learning on the points that adjustments of parameters are easy and paradoxical data-relations are permitted.

The paper presents the design of municipal creditworthiness parameters. Further, a model is designed based on Learning Vector Quantization neural networks for municipal creditworthiness classification. The model is composed of Kohonen's Self-organizing Feature Maps (unsupervised learning) whose outputs represent the input of the Learning Vector Quantization neural networks (supervised learning).

The back-propagation (BP) network and the Kohonen self-organizing feature map, selected as the representative types for the supervised and unsupervised artificial neural networks (ANN) respectively, are compared in terms of prediction accuracy in the area of bankruptcy prediction. Discriminant analysis and logistic regression are also performed to provide performance benchmarks. The findings suggest that the BP network is a better choice when a target vector is available. Advantages as well as limitations of the studied methods are also identified and discussed.

Decomposition of interacting features using a Kohonen self-organizing feature map neural network

One of the prerequisites of effective operation of multiphase flow systems is the need to identify flow regimes. A number of flow regime classification models have been reported in the literature based on the subjective and ambiguous interpretation of visual observations. With the advent of microprocessors and recent advances in signal processing, pattern recognition and neural network techniques, we are now on the threshold of finding more objective and quantitative technique for flow regime identification and flowrate measurement in multiphase systems. This paper presents a technique for the classification of flow regimes in horizontal air-water two-phase flow lines by the application of the Kohonen self-organising feature map (KSOFM).The principle of the technique rests on the characterisation and classification of turbulent pressure signals in relation with flow regimes. Characterisation means deriving stochastic features from time domain signals which are likely to be associated with specific physico-fluid dynamic groups based on space-time average values. Classification means defining the boundaries of the fluid regimes as represented by the fluid dynamic groups.KSOFM is known to be capable of organising patterns from an arbitrary n-dimension space into lower dimension space and preserving the topological relationships among the features. In this case, a KSOFM with 8*8 neurons was selected and trained with stochastic features derived from the absolute pressure signals obtained from a range of flow regimes. The resulting feature map succeeded in organising samples from similar flow regimes into same or adjacent neurons and separating samples from different flow regimes into different areas.The samples used in the tests were obtained in laboratory measurements conducted in a horizontal 2 inch diameter air-water flow line. KSOFM identified the flow regimes observed visually as bubbly, slug and wavy/stratified flow regimes as well as the more elusive transition regimes. For future work, it is proposed that the KSOFM offers a preprocessing capability in separating training regions into sub-regions where within each sub-region other supervised network models, such as the back propagation neural networks can be trained to identify primary quantities of interest such as average mass/volumetric flow rates of individual phases.

The location of a pixel in feature space is a function of its thematic composition. The latter is central to an image classification analysis, notably as an input (e.g., training data for a supervised classifier) and/or an output (e.g., predicted class label). Whether as an input to or output from a classification, little if any information beyond a class label is typically available for a pixel. The Kohonen self-organising feature map (SOFM) neural network however offers a means to both cluster together spectrally similar pixels that can be allocated suitable class labels and indicate relative thematic similarity of the clusters generated. Here, the thematic composition of pixels allocated to clusters represented by individual SOFM output units was explored with two remotely sensed data sets. It is shown that much of the spectral information of the input image data is maintained in the production of the SOFM output. This output provides a topologically structured representation of the image data, allowing spectrally similar pixels to be grouped together and the similarity of different clusters to be assessed. In particular, it is shown that the thematic composition of both pure and mixed pixels can be characterised by a SOFM. The location of the output unit in the output layer of the SOFM associated with a pixel conveys information on its thematic composition. Pixels in spatially close output units are more similar spectrally and thematically than those in more distant units. This situation also enables specific sub-areas of interest in the SOFM output space and/or feature space to be identified. This may, for example, provide a means to target efforts in training data acquisition for supervised classification as the most useful training cases may have a tendency to lie within specific sub-areas of feature space.

An investigation of local energy surface detection integrated with neural network techniques for image segmentation is presented, as applied in the feature extraction of chromosomes from image datasets obtained using an experimental confocal microscope. Use of the confocal microscope enables biologists to observe dividing cells (living or preserved) within a three-dimensional (3-D) volume, that can be visualised from multiple aspects, allowing for increased structural insight. The Nomarski differential interference contrast mode used for imaging translucent specimens, such as chromosomes, produces images not suitable for volume rendering. Segmentation of the chromosomes from this data is, thus, necessary. A neural network based on competitive learning, known as Kohonen's self-organizing feature map (SOFM) was used to perform segmentation, using a collection of statistics or features defining the image. Our past investigation showed that standard features such as the localized mean and variance of pixel intensities provided reasonable extraction of objects such as mitotic chromosomes, but surface detail was only moderately resolved. In this current work, a biologically inspired feature known as local energy is investigated as an alternative image statistic based on phase congruency in the image. This, along with different combinations of other image statistics, is applied in a SOFM, producing 3-D images exhibiting vast improvement in the level of detail and clearly isolating the chromosomes from the background. Index Terms-DIC, differential interference contrast, feature extraction, feature space, image segmentation, local energy, Morlet wavelet, phase congruency, self organizing feature map, SOFM.

Research on artificial neural networks (ANNs) has been carried out for more than five decades. A renewed interest appeared in the 80's with the finding of powerful models like J. Hopfield's recurrent networks, T. Kohonen's self-organizing feature maps, and the back-propagation rule. At that time, there was no platform that was at the same time versatile enough for any ANN model to be implemented and fast enough to solve large problems. Super-computers were the sole exception to this rule, but were prohibitively expensive for most applications. However, both research scientists and application engineers clearly identified the need for such a computing power. This triggered many projects in the field. In parallel, research on multi-processor systems started during the 60's. Systolic arrays have been proposed in 1979 as a means to fully exploit the possibilities of VLSI. Two previous theses, by F. Blayo and C. Lehmann, have studied the use of bi-dimensional systolic arrays for neural computation. At first, the presented system, called GENES, has been designed for the Hopfield model. Extensions to other ANNs have also been proposed. The goal of the present thesis is to study, design, build, and analyze an efficient accelerator for neural computation. In a first step, the GENES architecture has been extended towards generality and efficiency. This includes a thorough analysis of ANN models, of other neural computers, and of previous GENES implementations. The result of this work is the GENES IV integrated circuit, whose architecture has been co-designed by P. Ienne and the author. The main part of this thesis discusses the architecture, the design and the analysis of the MANTRA I machine, a neural computer based on a GENES IV array with up to 40 × 40 processing elements (PEs). The delta rule (and hence the Perceptron and Adaline rules), the back-propagation rule, the Hopfield model, and the Kohonen model can be implemented on this system. Although not a generic system, such a machine may be regarded as a multi-model neural computer. A prototype has been running for a year and is used daily by software designers. Several novel features distinguish the MANTRA I machine from other neural systems. First, it belongs to the few existing neural computers, contrary to the majority of implementations, which are specific to an application or to an algorithm. The machine does not hard-wire any algorithm, but provides the necessary primitives to implement the target models. This is a key feature for research, since several algorithms or versions of an algorithm can be tested on a problem. It is an important aspect for applications as well, because different ANN models are often cascaded to solve a problem. The GENES IV array — that is, the computing core of the MANTRA I machine — features synapse-level parallelism (i.e., one real or virtual PE is allocated per synapse or neural connection), while most other systems exploit only neuron-level parallelism (i.e., one PE per neuron). Hence, this system aims at a much finer parallelism grain and is well suited for massively parallel architectures. The problem size that can be computed by a neural accelerator should not be limited by the hardware (except for memory size). Therefore, it is essential to support time-sharing of PEs. On the MANTRA I machine, this is achieved by the concept of virtual arrays. Matrices are divided into sub-matrices that can be mapped onto the physical array, which is then time-shared among them. An efficient mechanism has been implemented to swap sub-matrices in background, while some other computation is performed. Since systolic arrays are pipelined systems, it is important to avoid emptying and re-filling them too often, in order to keep the hardware utilization rate high. Therefore, a systolic instruction flow has been implemented, so that each instruction follows the data for which it has been issued. Like any SIMD system, the MANTRA I machine is composed of a parallel or SIMD module and a control module. The SIMD module consists of the GENES IV array and a set of dedicated units designed for the computation that scales with the number of neurons and would poorly fit on a bi-dimensional array. A complex system of FIFOs and memories sustains the required input/output streams for the systolic array. The control module is a complete SISD system. Its tasks are (1) to control the SIMD module by dispatching instructions, (2) to manage data input and output, (3) to communicate with the external world, and (4) to perform data pre- and post-processing. The SIMD instructions are of the very long instruction word (VLIW) type. Synchronization between the two modules is achieved by an instruction FIFO. The performance of the MANTRA I machine has been analyzed using the delta rule. Measurements show that the sustained performance is very close to its peak value, as long as the problem fits in the memory banks connected to the GENES IV array. Experiments have also been run to investigate the impact of the constraints imposed by the hardware on the convergence of algorithms. Finally, the use of systolic arrays as neural accelerators is discussed in the light of the experience acquired with the GENES IV array and the MANTRA I machine. The weaknesses of the machine are analyzed, and several solutions are proposed to avoid them in a future design. A general discussion of the future of neural computers concludes this thesis.

QuestionCan a Kohonen's self‐organizing map, which is robust to non‐linear relationships between variables and their non‐normal distributions, be effective in patterning the data on plant communities investigated with the classical Braun‐Blanquet phytosociological method based on the ordinal scale? Does the application of the self‐organizing map make it possible to obtain new information from the analysed plant communities when compared to the Braun–Blanquet method alone?LocationThe Babsk nature reserve, Central Poland.MethodsThe analysed data were derived from two separate series of phytosociological studies on plant communities dating from periods 31 yr apart (1960 and 1991). The data consisted of 24 quantitative sampling lists of plant species (=phytosociological releves). The releves were analysed by the application of a Kohonen's self‐organizing map and the indicator value (IndVal).ResultsThe transformations from Querco roboris–Pinetum to Tilio cordatae–Carpinetum betuli in the vegetation in the Babsk nature reserve over 31 yr were determined precisely. Valuable new data were also obtained on: (1) significant associations of individual plant species with the previous and recent phytocoenoses, i.e. diagnostic groups of plant species for each phytocoenosis; (2) abiotic conditions (determined retrospectively in the two study periods on the basis of ecological indicators for vascular plants); and (3) the concordance of phytocoenoses with the biotope.ConclusionsThe Kohonen's self‐organizing map method and the IndVal made it possible to efficiently identify abiotic and biotic patterns for plant communities on the basis of the data expressed in the conventional Braun‐Blanquet scale.

Proposes a pattern classification method for remote sensing data based on neural network theory. From Kohonen's self-organizing feature maps, training areas for each pattern are selected. Using the back propagation algorithm, the layered neural network is trained such that the training patterns can be classified within a level. The experiments on LANDSAT TM data show that this approach produces excellent classification results compared with the conventional Bayesian approach.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Identification of lithofacies using Kohonen self-organizing maps

A neural piecewise linear classifier, based on the Kohonen learning vector quantization (LVQ2) and the Kohonen self-organizing feature map is proposed. The classifier has two stages and a feedback loop. In the first stage, the Kohonen self-organizing feature map network is used to find the approximate position of the prototype vectors for each class. In the second stage, the Kohonen LVQ2 supervised learning algorithm is used to fine-tune the position of the approximate prototype vectors. The accuracy of the classifier is improved by adding an adaptive feedback scheme. Depending on the intrinsic complexity of the class distribution and overall partitioning of the space, the neural classifier automatically increases the number of neurons, improving the error performance. The classifier was tested on a set of high-dimensional real data obtained from ship images. The performance is compared with a piecewise linear tree classifier and a neural classifier. >

A multilayer neural nerwork model for the perception of rotational motion has been developed using Reichardt's motion detector array of correlation type, Kohonen's self-organized feature map and Schuster-Wagner's oscillating neural network. It is shown that the unsupervised learning could make the neurons on the second layer of the network tend to be self-organized in a form resembling columnar organization of selective directions in area MT of the primate's visual cortex. The output layer can interpret rotation information and give the directions and velocities of rotational motion. The computer simulation results are in agreement with some psychophysical observations of rotational perception. It is demonstrated that the temporal correlation between the oscillating neurons would be powerful for solving the "binding problem" of shear components of rotational motion.

This paper addresses the problem of visual assessment of clustering tendency in p-dimensional data using two extensions of Kohonen's self-organizing feature map (SOFM). We show that SOFM cell displays generally do not produce visual evidence that leads to good guesses about cluster substructure or data density even for 2-dimensional data. The two proposed extensions of SOFM improve the quality of displays and enable us to make better guesses about the existence of substructure in data.

A new technique suitable for reduction of the number of colors in a color image is presented in this article. It is based on the use of the image Principal Color Components (PCC), which consist of the image color components and additional image components extracted with the use of proper spatial features. The additional spatial features are used to enhance the quality of the final image. First, the principal colors of the image and the principal colors of each PCC are extracted. Three algorithms were developed and tested for this purpose. Using Kohonen self‐organizing feature maps (SOFM) as classifiers, the principal color components of each PCC are obtained and a look‐up table, containing the principal colors of the PCC, is constructed. The final colors are extracted from the look‐up table entries through a SOFM by setting the number of output neurons equal to the number of the principal colors obtained for the original image. To speed up the entire algorithm and reduce memory requirements, a fractal scanning subsampling technique is employed. The method is independent of the color scheme; it is applicable to any type of color images and can be easily modified to accommodate any type of spatial features. Several experimental and comparative results exhibiting the performance of the proposed technique are presented. © 2002 Wiley Periodicals, Inc. Int J Imaging Syst Technol 12, 117–127, 2002; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ima.10019