A new neural network classifier based on ART theory

Photometric clustering of regenerated plants of gladiolus by neural networks and its biological validation

A self-organizing artificial neural network (ANN) that exhibits deterministically reliable behavior to noise interference when the noise does not exceed a specified level of tolerance is presented. The complexity of the proposed ANN, in terms of neuron requirements versus stored patterns, increases linearly with the number of stored patterns and their dimensionality. The self-organization of the proposed ANN is based on the idea of competitive generation and elimination of attraction wells in the pattern space. The same neural network can be used both for pattern recognition and classification. The ANN has been tested successfully with pattern and signal recognition and classification paradigms. >

Mapping adaptive resonance theory onto ring and mesh architectures

The adaptive resonance theory (ART) suggests a solution to the stability-plasticity dilemma facing designers of learning systems, namely how to design a learning system that will remain plastic, or adaptive, in response to significant events and yet remain stable in response to irrelevant events. ART architectures are discussed that are neural networks that self-organize stable recognition codes in real time in response to arbitrary sequences of input patterns. Within such an ART architecture, the process of adaptive pattern recognition is a special case of the more general cognitive process of hypothesis discovery, testing, search, classification, and learning. This property opens up the possibility of applying ART systems to more general problems of adaptively processing large abstract information sources and databases. The main computational properties of these ART architectures are outlined and contrasted with those of alternative learning and recognition systems.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Adaptive Resonance Theory (ART) neural networks model real-time prediction, search, learning, and recognition. ART networks function both as models of human cognitive information processing [1,2,3] and as neural systems for technology transfer [4]. A neural computation central to both the scientific and the technological analyses is the ART matching rule [5], which models the interaction between top-down expectation and bottom-up input, thereby creating a focus of attention which, in turn, determines the nature of coded memories. Sites of early and ongoing transfer of ART-based technologies include industrial venues such as the Boeing Corporation [6] and government venues such as MIT Lincoln Laboratory [7]. A recent report on industrial uses of neural networks [8] states: ”[The] Boeing ... Neural Information Retrieval System is probably still the largest-scale manufacturing application of neural networks. It uses [ART] to cluster binary templates of aeroplane parts in a complex hierarchical network that covers over 100,000 items, grouped into thousands of self-organised clusters. Claimed savings in manufacturing costs are in millions of dollars per annum.” At Lincoln Lab, a team led by Waxman developed an image mining system which incorporates several models of vision and recognition developed in the Boston University Department of Cognitive and Neural Systems (BU/CNS). Over the years a dozen CNS graduates (Aguilar, Baloch, Baxter, Bomberger, Cunningham, Fay, Gove, Ivey, Mehanian, Ross, Rubin, Streilein) have contributed to this effort, which is now located at Alphatech, Inc. Customers for BU/CNS neural network technologies have attributed their selection of ART over alternative systems to the model’s defining design principles. In listing the advantages of its THOT(r) technology, for example, American Heuristics Corporation (AHC) cites several characteristic computational capabilities of this family of neural models, including fast on-line (one-pass) learning, “vigilant” detection of novel patterns, retention of rare patterns, improvement with experience, ”weights [which] are understandable in real world terms,” and scalability (www.heuristics.com). Design principles derived from scientific analyses and design constraints imposed by targeted applications have jointly guided the development of many variants of the basic networks, including fuzzy ARTMAP [9], ART-EMAP [10], ARTMAP-IC [11], Gaussian ARTMAP [12], and distributed ARTMAP [3,13]. Comparative analysis of these systems has led to the identification of a default ARTMAP network, which features simplicity of design and robust performance in many application domains [4,14]. Selection of one particular ARTMAP algorithm is intended to facilitate ongoing technology transfer. The default ARTMAP algorithm outlines a procedure for labeling an arbitrary number of output classes in a supervised learning problem. A critical aspect of this algorithm is the distributed nature of its internal code representation, which produces continuous-valued test set predictions distributed across output classes. The character of their code representations, distributed vs. winner-take-all, is, in fact, a primary factor differentiating various ARTMAP networks. The original models [9,15] employ winner-take-all coding during training and testing, as do many subsequent variations and the majority of ART systems that have been transferred to technology. ARTMAP variants with winner-take-all (WTA) coding and discrete target class predictions have, however, shown consistent deficits in labeling accuracy and post-processing adjustment capabilities. The talk will describe a recent application that relies on distributed code representations to exploit the ARTMAP capacity for one-to-many learning, which has enabled the development of self-organizing expert systems for multi-level object grouping, information fusion, and discovery of hierarchical knowledge structures. A pilot study has demonstrated the network’s ability to infer multi-level fused relationships among groups of objects in an image, without any supervised labeling of these relationships, thereby pointing to new methodologies for self-organizing knowledge discovery.

Reservoirs and the lateral seal of stratigraphic traps are controlled by the depositional environment or diagenesis. The recognition of facies and lithology from seismic attributes is an effective approach for identifying stratigraphic traps related to the depositional environment. In this paper, the occurrence of stratigraphic traps related to depositional environment in Permian aeolian clastics and Jurassic carbonate-evaporites was studied. To identify these stratigraphic traps, multiple seismic attributes were classified using supervised and unsupervised artificial neural networks (ANNs), which allowed the recognition of seismic facies and lithology. Neural networks are a powerful classification technique, which incorporates multiple attributes into a number of classes to identify sedimentary facies. Two algorithms comprising supervised and unsupervised neural networks are commonly implemented. With a supervised learning algorithm, prior information such as typical facies at the control wells are required to train the multilayer perceptron (MLP) network. With an unsupervised algorithm, only seismic data is input to the neural network, and competitive-learning techniques are employed to classify or self-organize the data based on its internal characteristics. Without prior information, the output classes are not labeled with lithofacies. According to the availability of prior information, supervised and unsupervised learning were applied to recognize dune-playa and carbonate-evaporite combinations, respectively. To characterize the depositional environments, joint interpretation with a geological model is necessary for both supervised and unsupervised classification. Two major findings have been derived from this work. First, the learning technology based on ANNs is effective to recognize sedimentary facies. The microfacies and lithologies identified by both supervised and unsupervised ANNs are very consistent with the drilled wells. Second, the recognition of depositional facies and lithology can characterize the stratigraphic traps in the study areas. Lateral seal plays a key role in stratigraphic traps. Playa siltstone and tight lagoonal limestone constitute the lateral seal in dune-playa and carbonate-evaporite combinations, respectively.

While learning algorithms have been used for astronomical data analysis, the vast majority of those algorithms have used supervised learning. In a continuation of the work described in Young et ah [18] we examine the use of unsupervised learning for this task with two types of Adaptive Resonance Theory (ART) neural networks. Using synthetic astronomical data from SkyMaker[2], [3] which was designed to mimic the dynamic range of the CTI-[14] telescope, we compared the ability of the ART-1 neural network[4] and the ART-1 neural network with category theoretic modiflcation[9], [11] to detect regions of interest and to characterize noise. We show a difference in the geometries of the templates created by each architecture. We also show an analysis of the two architectures over a range of parameter settings. The results provided show that ART neural networks and unsupervised learning algorithms in general should not be overlooked for astronomical data analysis.

With the increasing emphasis on customer demands by organizations, customer requirements management has secured a crucial position during the early stage of product conceptualization. However, it is still of scarcity to have a systematic strategy for soliciting customer requirements effectively. For this purpose, a customer-oriented approach has accordingly been proposed in this study. This prototype system is composed of two interrelated components, namely, the customer requirements elicitation (CRE) and customer/marketing analysis (CMA) modules. In the context of the prototype system, the laddering technique has been employed to enable the customer requirements elicitation via the so-called customer attributes hierarchy (CAH). In addition, the adaptive resonance theory (particularly ART2) neural network has also been applied as a toolkit for further customer and marketing analysis. A case study on mobile hand phone design is used as an example to validate and illustrate the prototype system.

The adaptive resonance theory 2 (ART2) neural network exhibits several properties which can be useful in the data mining and which are lacking in most other neural networks. But ART2 has deficiencies that the categories clustered by ART2 are very mutable to slight changes in training conditions. An improved ART2 with enhanced triplex matching mechanism, named as ETM-ART2, is presented to redress the deficiencies. Several tests results show that ETM-ART2 performs better than classic ART2 when applied to clustering tasks. It is an effective improved algorithm and can be applied to a wide variety of problems.

An efficient approach for electric load forecasting using distributed ART (adaptive resonance theory) &amp; HS-ARTMAP (Hyper-spherical ARTMAP network) neural network

This study presents a simplified fuzzy ARTMAP (SFAM) for different classification applications. The proposed SFAM model is synergy of fuzzy logic and adaptive resonance theory (ART) neural networks. SFAM is supervised network consisting of two layers (Fuzzy ART and Inter ART) that build constant classification groups in answer to series of input patterns. Fuzzy ART layer takes a series of input patterns and relate them to output classes. Inter ART layer functions in such a way that it raises the vigilance parameter of fuzzy ART layer. By combining this two layers, SFAM is capable to perform classification very efficiently and giving very high performances. Lastly, the SFAM model is applied to different simulations. The simulation results obtained for the three different datasets: Iris, Wisconsin breast cancer and wine dataset prove that SFAM model has better performance results than other models for these classification applications.

The internal representation of the categories in Adaptive Resonance Theory (ART) neural networks can greatly affect the quality and compactness of the discovered clusters. Dual vigilance thresholds have been shown to yield a significant improvement in Fuzzy ART performance and allow it to retrieve arbitrarily shaped clusters while maintaining the important advantage of a simple architecture and implementation. In this study, we examine the use of Hypersphere ART within the same dual vigilance architecture, thereby presenting the Dual Vigilance Hypersphere ART (DVHA). We conduct an extensive comparison between 6 different ART-based approaches across a set of 30 benchmark datasets, using two different input ordering methods and 30 repeated runs. We find that DVHA ranks better than Dual Vigilance Fuzzy ART (DVFA) on average for many datasets, both in terms of performance and network compactness. Furthermore, although another multi-category-based architecture showed statistically superior results when the inputs are shuffled, we found no statistical difference in performance when the input was pre-processed using the visual assessment of cluster tendency (VAT), while generally being much simpler to implement and less computationally demanding. These findings make DVHA a viable alternative to its Fuzzy ART counterpart, and a simpler alternative to the other studied multi-category-based approaches in cases where resources are limited, such as embedded and hardware-based applications, provided that the input can be preprocessed using VAT.

Adaptive Resonance Theory (ART), developed by Stephen Grossberg and Gail Carpenter, has attracted some enthusiasts worldwide as a promising approach of artificial neural networks (ANN) for pattern recognition and classification tasks. However, this technology is still not popular among IT professionals and its adoption for data mining applications has not spread out as one could expect. We believe that one reason for this fact is the lack of knowledge about the ART ANN behavior as the parameters change in space. To explore this idea, we executed almost eight hundred experimental tests to map ART1, ART2 and Fuzzy-ART configuration parameters. Some analyses have been done and the results can help system analysts deal with this kind of ANN and explore its complete power. Experimental data, referring to simulated and real databases, have shown that dependent parameter ~ could be clearly defined in Euclidean space R2 as a function of the vigilance parameter p and other patterns and network parameters in ART1, ART2 and Fuzzy-ART models. These results set numerical boundaries up that are useful for data mining application designers using such technology. For instance, the behavior of normalized parameter < as a function of p shows that it is possible to overcome problems with binary coding in ART1 by imposing higher values of p to get complete featured cluster vectors and attenuate (or eliminate) the “bit flood effect” of “O” and” 1“ observed in “1of-N” or “thermometer” coding. An important conclusion is that ART1, ART2 and Fuzzy-ART neural networks, as many other ANN approaches, can support data mining applications based on unsupervised learning (for on line systems) overcoming the stability-plasticity dilemma present in other ANN models. © 2002 WIT Press, Ashurst Lodge, Southampton, SO40 7AA, UK. All rights reserved. Web: www.witpress.com Email witpress@witpress.com Paper from: Data Mining III, A Zanasi, CA Brebbia, NFF Ebecken & P Melli (Editors). ISBN 1-85312-925-9

The second version of the simplified ART-based artificial neural network, named SARTNN2, is presented. The SARTNN2 model, which is made of an attentional and of an orienting subsystem, detects statistical regularities in a random sequence of multivalued input patterns. The SARTNN2 attentional subsystem consists of a flat, self-organizing Kohonen's artificial neural network which exploits a bubble competitive strategy. Both the attentional and the orienting subsystems exploit a new metric, named normalized vector distance (NVD), which provides a normalized measurement of the intervector distance between a multivalued vector pair. The NVD measurements, which are per cent values, feature adaptability to local statistics. SARTNN2 inherits the typical KANN plasticity in addition to the peculiar ART stability. Only two user-defined parameters, both having an intuitive physical meaning, are necessary for SARTNN2 to operate. The SARTNN2 model has been tested as a clustering algorithm in the unsupervised analysis of satellite imagery.

In this paper, two evolutionary artificial neural network (EANN) models that are based on integration of two supervised adaptive resonance theory (ART)-based artificial neural networks with a hybrid genetic algorithm (HGA) are proposed. The search process of the proposed EANN models is guided by a knowledge base established by ART with respect to the training data samples. The EANN models explore the search space for “coarse” solutions, and such solutions are then refined using the local search process of the HGA. The performances of the proposed EANN models are evaluated and compared with those from other classifiers using more than ten benchmark data sets. The applicability of the EANN models to a real medical classification task is also demonstrated. The results from the experimental studies demonstrate the effectiveness and usefulness of the proposed EANN models in undertaking pattern classification problems.