An Efficient Two-Level SOMART Document Clustering Through Dimensionality Reduction

Photometric clustering of regenerated plants of gladiolus by neural networks and its biological validation

Availability of large full-text document collections in electronic form has created a need for tools and techniques that assist users in organizing these collections. Document clustering is one of the popular methods used for this purpose. The Self-organizing map (SOM), an unsupervised algorithm for clustering and topographic mapping, has shown promising results in this task. Most of the existing SOM techniques rely on a “bag of words” document representation. Each word in the document is considered as a separate feature, ignoring the word order. In this chapter we investigate the use of phrases rather than words as document features for document clustering. We present a phrase grammar extraction technique, and use the extracted phrases as features in two different document clustering algorithms, self-organizing map (SOM) and hierarchical self-organizing map (HSOM). We present results of clustering documents from the REUTERS corpus and show an improvement in the clustering performance evaluated using the entropy and F-measure.

Automated detection of lane-blocking freeway incidents using artificial neural networks

This study presents a simplified fuzzy ARTMAP (SFAM) for different classification applications. The proposed SFAM model is synergy of fuzzy logic and adaptive resonance theory (ART) neural networks. SFAM is supervised network consisting of two layers (Fuzzy ART and Inter ART) that build constant classification groups in answer to series of input patterns. Fuzzy ART layer takes a series of input patterns and relate them to output classes. Inter ART layer functions in such a way that it raises the vigilance parameter of fuzzy ART layer. By combining this two layers, SFAM is capable to perform classification very efficiently and giving very high performances. Lastly, the SFAM model is applied to different simulations. The simulation results obtained for the three different datasets: Iris, Wisconsin breast cancer and wine dataset prove that SFAM model has better performance results than other models for these classification applications.

The internal representation of the categories in Adaptive Resonance Theory (ART) neural networks can greatly affect the quality and compactness of the discovered clusters. Dual vigilance thresholds have been shown to yield a significant improvement in Fuzzy ART performance and allow it to retrieve arbitrarily shaped clusters while maintaining the important advantage of a simple architecture and implementation. In this study, we examine the use of Hypersphere ART within the same dual vigilance architecture, thereby presenting the Dual Vigilance Hypersphere ART (DVHA). We conduct an extensive comparison between 6 different ART-based approaches across a set of 30 benchmark datasets, using two different input ordering methods and 30 repeated runs. We find that DVHA ranks better than Dual Vigilance Fuzzy ART (DVFA) on average for many datasets, both in terms of performance and network compactness. Furthermore, although another multi-category-based architecture showed statistically superior results when the inputs are shuffled, we found no statistical difference in performance when the input was pre-processed using the visual assessment of cluster tendency (VAT), while generally being much simpler to implement and less computationally demanding. These findings make DVHA a viable alternative to its Fuzzy ART counterpart, and a simpler alternative to the other studied multi-category-based approaches in cases where resources are limited, such as embedded and hardware-based applications, provided that the input can be preprocessed using VAT.

The adaptive resonance theory 2 (ART2) neural network exhibits several properties which can be useful in the data mining and which are lacking in most other neural networks. But ART2 has deficiencies that the categories clustered by ART2 are very mutable to slight changes in training conditions. An improved ART2 with enhanced triplex matching mechanism, named as ETM-ART2, is presented to redress the deficiencies. Several tests results show that ETM-ART2 performs better than classic ART2 when applied to clustering tasks. It is an effective improved algorithm and can be applied to a wide variety of problems.

Adaptive Resonance Theory (ART) neural networks model real-time prediction, search, learning, and recognition. ART networks function both as models of human cognitive information processing [1,2,3] and as neural systems for technology transfer [4]. A neural computation central to both the scientific and the technological analyses is the ART matching rule [5], which models the interaction between top-down expectation and bottom-up input, thereby creating a focus of attention which, in turn, determines the nature of coded memories. Sites of early and ongoing transfer of ART-based technologies include industrial venues such as the Boeing Corporation [6] and government venues such as MIT Lincoln Laboratory [7]. A recent report on industrial uses of neural networks [8] states: ”[The] Boeing ... Neural Information Retrieval System is probably still the largest-scale manufacturing application of neural networks. It uses [ART] to cluster binary templates of aeroplane parts in a complex hierarchical network that covers over 100,000 items, grouped into thousands of self-organised clusters. Claimed savings in manufacturing costs are in millions of dollars per annum.” At Lincoln Lab, a team led by Waxman developed an image mining system which incorporates several models of vision and recognition developed in the Boston University Department of Cognitive and Neural Systems (BU/CNS). Over the years a dozen CNS graduates (Aguilar, Baloch, Baxter, Bomberger, Cunningham, Fay, Gove, Ivey, Mehanian, Ross, Rubin, Streilein) have contributed to this effort, which is now located at Alphatech, Inc. Customers for BU/CNS neural network technologies have attributed their selection of ART over alternative systems to the model’s defining design principles. In listing the advantages of its THOT(r) technology, for example, American Heuristics Corporation (AHC) cites several characteristic computational capabilities of this family of neural models, including fast on-line (one-pass) learning, “vigilant” detection of novel patterns, retention of rare patterns, improvement with experience, ”weights [which] are understandable in real world terms,” and scalability (www.heuristics.com). Design principles derived from scientific analyses and design constraints imposed by targeted applications have jointly guided the development of many variants of the basic networks, including fuzzy ARTMAP [9], ART-EMAP [10], ARTMAP-IC [11], Gaussian ARTMAP [12], and distributed ARTMAP [3,13]. Comparative analysis of these systems has led to the identification of a default ARTMAP network, which features simplicity of design and robust performance in many application domains [4,14]. Selection of one particular ARTMAP algorithm is intended to facilitate ongoing technology transfer. The default ARTMAP algorithm outlines a procedure for labeling an arbitrary number of output classes in a supervised learning problem. A critical aspect of this algorithm is the distributed nature of its internal code representation, which produces continuous-valued test set predictions distributed across output classes. The character of their code representations, distributed vs. winner-take-all, is, in fact, a primary factor differentiating various ARTMAP networks. The original models [9,15] employ winner-take-all coding during training and testing, as do many subsequent variations and the majority of ART systems that have been transferred to technology. ARTMAP variants with winner-take-all (WTA) coding and discrete target class predictions have, however, shown consistent deficits in labeling accuracy and post-processing adjustment capabilities. The talk will describe a recent application that relies on distributed code representations to exploit the ARTMAP capacity for one-to-many learning, which has enabled the development of self-organizing expert systems for multi-level object grouping, information fusion, and discovery of hierarchical knowledge structures. A pilot study has demonstrated the network’s ability to infer multi-level fused relationships among groups of objects in an image, without any supervised labeling of these relationships, thereby pointing to new methodologies for self-organizing knowledge discovery.

Distributed memory and localized memory in neural networks

An efficient approach for electric load forecasting using distributed ART (adaptive resonance theory) & HS-ARTMAP (Hyper-spherical ARTMAP network) neural network

Document clustering is a very useful application in recent days especially with the advent of the World Wide Web. Most of the existing document clustering algorithms either produce clusters of poor quality or are highly computationally expensive. In this paper we propose a document-clustering algorithm, KMART, that uses an unsupervised fuzzy adaptive resonance theory (fuzzy-ART) neural network. A modified version of the fuzzy ART is used to enable a document to be in multiple clusters. The number of clusters is determined dynamically. Some experiments are reported to compare the efficiency and execution time of our algorithm with other document-clustering algorithm like fuzzy c-means. The results show that KMART is both effective and efficient.

ART-2 is a self-organized and unsupervised artificial neural network constructed from adaptive resonance theory which can be used to classify continuous active data. We have found that the theory is limited of the same phase data with different amplitudes and insensitivity to gradual change data during the simulation of data classified with ART-2 neural network. Therefore, we propose a new neural network model based on adaptive resonance theory. We provide the model construction and relevant algorithm as well as the comparison with ART-2.

Clustering methods have been widely applied to the fault diagnosis of mechanical system, but the characteristic that the number of cluster needs to be determined in advance limits the application range of the method. In this paper, a novel clustering method combining the adaptive resonance theory (ART) with the similarity measure based on the Yu’s norm is presented and applied to the fault diagnosis of rolling element bearings, which can be adaptive to generate the number of cluster by the vigilance parameter test. Time-domain features, frequency-domain features, and time series model parameters are extracted to demonstrate the fault-related information about the bearings, and then considering the irrelevance or redundancy of some features many salient features are selected by an improved distance discriminant technique and input into the proposed clustering method to diagnose the faults of bearings. The experiment results confirmed that the proposed clustering method can diagnose the fault categories accurately and has better diagnosis performance compared with fuzzy ART and Self-Organizing Feature Map (SOFM).

We combine neural networks (NNs) and hidden Markov models (HMMs) techniques in order to obtain the lithology identification of wells situated in the Triasic province (Sahara). For the same aim, two systems based on adaptive resonance theory (ART), ART1 and fuzzy ART, are also developed. Our objective is to facilitate the work of the geological experts by permitting them to obtain quickly the structure and the nature of lands around the drilling. Lithology identification supplies qualitative information about rocks. Lithofacies represent rock textures and are important components of hydrocarbon reservoir characterisation. In this paper, we show that it is interesting to combine the respective capacities of the HMMs and NNs to produce a new effective hybrid models that draw their source in the two formalisms and can provide us a more reliable reservoir model. Comparisons are established to show that the results obtained by the NN-HMM hybrid system are close to those obtained by the fuzzy ART approach applied to the same borehole with the same well logs

The article presents neural network for recognition of driving strategies based on interactions between drivers in road traffic. It analyzes the architecture of the model implemented as a self-organizing map (SOM), consisting of a group of neural networks based on radial basis functions (RBF). It is a training model grounded in the biological foundations of artificial neural networks, in which the training set should consist exclusively of input vectors; wherein the network training algorithm adjusts itself the network's weights to obtain consistent output vectors (i.e. to make presenting sufficiently close input vectors result into the same outputs).\nThe article presents the results of using a new generation of the neural network developed by us, which includes an adaptive learning algorithm to reduce the effect of re-training (overfitting) and false recognition, as well as to improve the determination of the boundaries between clusters.\nThe aim of the research is to outline architecture and structure of the neural network model that allows recognizing strategical characteristics of driving and can identify strategies of interactions between vehicles (their drivers) in road traffic as well as identify behavioral patterns This paper considers driving strategies that characterize the interaction of dyads of vehicles (drivers) moving in road traffic. The research results show that the SOM RBF neural networks can recognize and classify types of interactions in road traffic based on modelling of the analysis of vehicle movement trajectories. Experimental results demonstrate the neural networks architecture and networks learning involving 400 iterations of streaming the training data representing 500 possible simulated interaction situations. This paper presents a novel neural network model for recognition of drivers' behaviour patterns and for classification of driving strategies into five general classes: (1) competition strategy, (2) contest strategy, (3) evasion strategy, (4) compromise strategy, and (5) active confrontation strategy. This neural network has demonstrated a high rate of recognition and concise clusterization of similar driving strategies. The key contribution of this paper: it proposes a neural network model based on Kohonen's Self-Organizing Map (SOM) for detecting drivers' behaviours from vehicle movement patterns - driving strategies - instead of monitoring driver's specific activities.

A fault detection and diagnosis method for the hydraulic servo system based on adaptive threshold and self-organizing map (SOM) neural network is proposed in this study. The nonlinear, time-varying, fluid-solid coupling properties of the hydraulic servo system are considered. Fault detection is realized based on a two-stage radial basis function (RBF) neural network model. The first-stage RBF neural network is adopted as a fault observer for the hydraulic servo system; the residual error signal is generated by comparing the estimated observer output with the actual measurements. To overcome the drawback of false alarms when the traditional fixed fault threshold is used, an adaptive threshold producer is established by the second-stage RBF neural network. Fault occurrence is detected by comparing the residual error signal with the adaptive threshold. When a system fault is detected, the SOM neural network is employed to implement fault classification and isolation by analyzing the features of the residual error signal. Three types of common faults are simulated to verify the performance and effectiveness of the proposed method. Experimental results demonstrate that the proposed method based on adaptive threshold and SOM neural network is effective in detecting and isolating the failure of the hydraulic servo system.