Evaluating quality of text clustering with ART1

Document clustering has been an important research area in recent years. However, most work on this subject has focused on batch processing in a static environment. For real world applications, online and incremental processing of highly dynamic data is required. Adaptive resonance theory (ART) neural networks possess several interesting properties that make them appealing as a potential solution to this problem. In this paper, we present preliminary experimental results that examine ART text clustering under several situations characteristic of real-life applications. We also compare our present results with work we have conducted previously on the batch static case, hence determining how clustering quality is affected by incremental processing.

This study presents a simplified fuzzy ARTMAP (SFAM) for different classification applications. The proposed SFAM model is synergy of fuzzy logic and adaptive resonance theory (ART) neural networks. SFAM is supervised network consisting of two layers (Fuzzy ART and Inter ART) that build constant classification groups in answer to series of input patterns. Fuzzy ART layer takes a series of input patterns and relate them to output classes. Inter ART layer functions in such a way that it raises the vigilance parameter of fuzzy ART layer. By combining this two layers, SFAM is capable to perform classification very efficiently and giving very high performances. Lastly, the SFAM model is applied to different simulations. The simulation results obtained for the three different datasets: Iris, Wisconsin breast cancer and wine dataset prove that SFAM model has better performance results than other models for these classification applications.

The internal representation of the categories in Adaptive Resonance Theory (ART) neural networks can greatly affect the quality and compactness of the discovered clusters. Dual vigilance thresholds have been shown to yield a significant improvement in Fuzzy ART performance and allow it to retrieve arbitrarily shaped clusters while maintaining the important advantage of a simple architecture and implementation. In this study, we examine the use of Hypersphere ART within the same dual vigilance architecture, thereby presenting the Dual Vigilance Hypersphere ART (DVHA). We conduct an extensive comparison between 6 different ART-based approaches across a set of 30 benchmark datasets, using two different input ordering methods and 30 repeated runs. We find that DVHA ranks better than Dual Vigilance Fuzzy ART (DVFA) on average for many datasets, both in terms of performance and network compactness. Furthermore, although another multi-category-based architecture showed statistically superior results when the inputs are shuffled, we found no statistical difference in performance when the input was pre-processed using the visual assessment of cluster tendency (VAT), while generally being much simpler to implement and less computationally demanding. These findings make DVHA a viable alternative to its Fuzzy ART counterpart, and a simpler alternative to the other studied multi-category-based approaches in cases where resources are limited, such as embedded and hardware-based applications, provided that the input can be preprocessed using VAT.

While learning algorithms have been used for astronomical data analysis, the vast majority of those algorithms have used supervised learning. In a continuation of the work described in Young et ah [18] we examine the use of unsupervised learning for this task with two types of Adaptive Resonance Theory (ART) neural networks. Using synthetic astronomical data from SkyMaker[2], [3] which was designed to mimic the dynamic range of the CTI-[14] telescope, we compared the ability of the ART-1 neural network[4] and the ART-1 neural network with category theoretic modiflcation[9], [11] to detect regions of interest and to characterize noise. We show a difference in the geometries of the templates created by each architecture. We also show an analysis of the two architectures over a range of parameter settings. The results provided show that ART neural networks and unsupervised learning algorithms in general should not be overlooked for astronomical data analysis.

The case retrieval model based on neural network is presented to enhance the efficiency and quality of retrieving case in the case-based reasoning system of the motorcycle intelligent design. In the retrieval model, the adaptive resonance theory neural network was used to dynamically cluster the cases in the case base to narrow the searching range. The back propagation neural network was applied to memory the index of cases to retrieve quickly the similar case from the narrowed case base. Thus the efficiency and quality of retrieving case are improved. Finally, an example of the plan selection of motorcycle general design was given. Its result was contrasted with that of case retrieval based on the nearest neighbor method to demonstrate the effectives of the case retrieval model. The research shows that it is practicable and effective using the adaptive resonance theory and BP neural network to modeling the reasoning mechanism.

A digital VLSI circuit design for an adaptive resonance theory (ART) neural network architecture, called the augmented ART-1 neural network (AARTI-NN) is presented. An ‘axon-synapse-tree’ structure is used to realize the activities of the short-term memories and reset subsystem. The long-term memory traces are implemented using NMOS transmission gales. PSpice circuit simulation was carried out to verify the design of a prototype, seven-node AARTI-NN. A clock-controlled delay element is included in the simulation to illustrate the functionality of the AARTI-NN. It is shown that the AARTI-NN node selection activities simulated from the circuit designed are identical to those described by the coupled differential equations governing the AARTI-NN.

With the increasing emphasis on customer demands by organizations, customer requirements management has secured a crucial position during the early stage of product conceptualization. However, it is still of scarcity to have a systematic strategy for soliciting customer requirements effectively. For this purpose, a customer-oriented approach has accordingly been proposed in this study. This prototype system is composed of two interrelated components, namely, the customer requirements elicitation (CRE) and customer/marketing analysis (CMA) modules. In the context of the prototype system, the laddering technique has been employed to enable the customer requirements elicitation via the so-called customer attributes hierarchy (CAH). In addition, the adaptive resonance theory (particularly ART2) neural network has also been applied as a toolkit for further customer and marketing analysis. A case study on mobile hand phone design is used as an example to validate and illustrate the prototype system.

A new method of texture classification comprising two processing stages, namely a low-level evolutionary feature extrac- tion based on Gabor wavelets and a high-level neural network based pattern recognition, is proposed. The design of these stages is motivated by the processes involved in the human visual system: low-level receptors responsible for early vision processing and the high-level cognition. Gabor wavelets are used as extractors of ''low- level'' features that feed the feature-adaptive adaptive resonance theory (ART) neural network acting as a high-level ''cognitive sys- tem.'' The novelty of the model developed in this paper lies in the use of a self-organizing input layer to the fuzzy ART. Evaluation of the model is performed by using natural textures, and results ob- tained show that the developed model is capable of performing the texture recognition task effectively. Applications of the developed model include the study of artificial vision systems motivated by the human visual system model. © 1997 SPIE and IS&T. (S1017-9909(97)00403-0)

With a focus on classification problem, in this paper, we present an integrated approach to improve the performance of classification using adaptive resonance theory (ART) neural network and logistic regression classifiers. In our approach, the neural network classifier is trained first and then regression analysis is applied to each individual class. In testing phase, the data is applied to the regression classifier and, if any deviation exists, the neural network classifier is retrained. The study reveals that effective data mining can be achieved by combining the power of neural networks with the rigor of more traditional statistical tools.

Adaptive Resonance Theory (ART) neural networks model real-time prediction, search, learning, and recognition. ART networks function both as models of human cognitive information processing [1,2,3] and as neural systems for technology transfer [4]. A neural computation central to both the scientific and the technological analyses is the ART matching rule [5], which models the interaction between top-down expectation and bottom-up input, thereby creating a focus of attention which, in turn, determines the nature of coded memories. Sites of early and ongoing transfer of ART-based technologies include industrial venues such as the Boeing Corporation [6] and government venues such as MIT Lincoln Laboratory [7]. A recent report on industrial uses of neural networks [8] states: ”[The] Boeing ... Neural Information Retrieval System is probably still the largest-scale manufacturing application of neural networks. It uses [ART] to cluster binary templates of aeroplane parts in a complex hierarchical network that covers over 100,000 items, grouped into thousands of self-organised clusters. Claimed savings in manufacturing costs are in millions of dollars per annum.” At Lincoln Lab, a team led by Waxman developed an image mining system which incorporates several models of vision and recognition developed in the Boston University Department of Cognitive and Neural Systems (BU/CNS). Over the years a dozen CNS graduates (Aguilar, Baloch, Baxter, Bomberger, Cunningham, Fay, Gove, Ivey, Mehanian, Ross, Rubin, Streilein) have contributed to this effort, which is now located at Alphatech, Inc. Customers for BU/CNS neural network technologies have attributed their selection of ART over alternative systems to the model’s defining design principles. In listing the advantages of its THOT(r) technology, for example, American Heuristics Corporation (AHC) cites several characteristic computational capabilities of this family of neural models, including fast on-line (one-pass) learning, “vigilant” detection of novel patterns, retention of rare patterns, improvement with experience, ”weights [which] are understandable in real world terms,” and scalability (www.heuristics.com). Design principles derived from scientific analyses and design constraints imposed by targeted applications have jointly guided the development of many variants of the basic networks, including fuzzy ARTMAP [9], ART-EMAP [10], ARTMAP-IC [11], Gaussian ARTMAP [12], and distributed ARTMAP [3,13]. Comparative analysis of these systems has led to the identification of a default ARTMAP network, which features simplicity of design and robust performance in many application domains [4,14]. Selection of one particular ARTMAP algorithm is intended to facilitate ongoing technology transfer. The default ARTMAP algorithm outlines a procedure for labeling an arbitrary number of output classes in a supervised learning problem. A critical aspect of this algorithm is the distributed nature of its internal code representation, which produces continuous-valued test set predictions distributed across output classes. The character of their code representations, distributed vs. winner-take-all, is, in fact, a primary factor differentiating various ARTMAP networks. The original models [9,15] employ winner-take-all coding during training and testing, as do many subsequent variations and the majority of ART systems that have been transferred to technology. ARTMAP variants with winner-take-all (WTA) coding and discrete target class predictions have, however, shown consistent deficits in labeling accuracy and post-processing adjustment capabilities. The talk will describe a recent application that relies on distributed code representations to exploit the ARTMAP capacity for one-to-many learning, which has enabled the development of self-organizing expert systems for multi-level object grouping, information fusion, and discovery of hierarchical knowledge structures. A pilot study has demonstrated the network’s ability to infer multi-level fused relationships among groups of objects in an image, without any supervised labeling of these relationships, thereby pointing to new methodologies for self-organizing knowledge discovery.

Selecting appropriate mechanism types that meet design requirements is a critical problem often encountered in conceptual design of mechanical system. A novel approach to mechanism coding is presented at first, in which the features of motion function and function quality for mechanism can be expressed respectively with a list of binary vectors. A retrieval approach to mechanism type selection is then proposed using adaptive resonance theory (ART) neural network. Under this approach, sets of binary vectors representing all mechanisms are fed into an ART1 network to structure mechanism clusters and a proper number of reference mechanisms can be received after a binary vector representing design requirements is fed into the pre-grouped network. Compared with other retrieval system, by adjusting the value of vigilance parameter, the designer can obtain an optimal mechanism or several satisfactory mechanisms more easily in terms of his design intent using this approach.

The adaptive resonance theory 2 (ART2) neural network exhibits several properties which can be useful in the data mining and which are lacking in most other neural networks. But ART2 has deficiencies that the categories clustered by ART2 are very mutable to slight changes in training conditions. An improved ART2 with enhanced triplex matching mechanism, named as ETM-ART2, is presented to redress the deficiencies. Several tests results show that ETM-ART2 performs better than classic ART2 when applied to clustering tasks. It is an effective improved algorithm and can be applied to a wide variety of problems.

A simple modification of the adaptive resonance theory (ART) neural network allows shift, scale and rotation invariant learning. The authors point out that this can be accomplished as a neural architecture by modifying the standard ART with hardwired interconnects that perform a Fourier-Mellin transform, and show how to modify the heuristics for efficient simulation of ART architectures to accomplish the additional innovation. Finally, they discuss the implementation of this in optoelectronic hardware, using a modification of the Van der Lugt optical correlator. >

Generalized Adaptive Resonance Theory (GART) is a neural network model based on the integration of Gaussian ARTMAP and the Generalized Regression Neural Network. As demonstrated in our previous wor...

Group incremental adaptive clustering based on neural network and rough set theory for crime report categorization