Controlling the spread of dynamic self-organising maps

We have been working on the application of Machining Learning in Metabolomics, Drug-Drug Interaction Discovery and Human Gait Recognition [1-5], profiling large data sets. Extraction of vital information about 1) plant metabolomics that can improve the environment and food quality, 2) in vitro neuronal network behavior patterns for various drugs that can be used to characterize drugs for brain deceases and 3) human gait patterns that can reveal various diseases were among these applications. We have demonstrated with considerable success in using unsupervised clustering techniques to analyze genetic and metabolomic data. This includes analysis of drought resistance in wheat [4] and microbial metagenomes [5].We introduced two methods: Near Unsupervised Learning (NUL) and Sub-sample Error Graphs (SEGs) [5] to analyze large amount of data. Self Organizing Map (SOM), which is one of the widely used Unsupervised Neural Networks, has been used as a data-mining tool due to its ability to map high dimensional data into a two dimensional feature map, which is expected to be topology preserving allowing users to visually identify clusters by their topological relationships in terms of their proximity on the map. The Growing Self Organizing Maps (GSOM) further allows the map size to be determined by the algorithm, which relies upon a user set parameter called Spread Factor (SF) [6]. The wide availability of GPUs for affordable prices allows faster comparison of various SOMs with different maps sizes taking away some of the advantages GSOM claimed. Further development of GSOM into a Dynamic SOM Tree exploits the possibility of varying SF to obtain multiple GSOMs from a small number of compact clusters to a large number of sparse clusters [7]. NUL methods can be applied on GSOM using a small number of labels that should be available for every class. This is however not realistic in some applications where the number of classes cannot be explicitly known. We propose a new method call Deep Near Unsupervised Learning (D-NUL), where Dynamic SOM tree is used instead of GSOM and the number of classes are not assumed to be known. The implementation of Dynamic SOM tree methods with varying SF on GPUs will make the computation with D-NUL possible for many problems in big data analytics.

The growing self-organizing map (GSOM) has been presented as an extended version of the self-organizing map (SOM), which has significant advantages for knowledge discovery applications. In this paper, the GSOM algorithm is presented in detail and the effect of a spread factor, which can be used to measure and control the spread of the GSOM, is investigated. The spread factor is independent of the dimensionality of the data and as such can be used as a controlling measure for generating maps with different dimensionality, which can then be compared and analyzed with better accuracy. The spread factor is also presented as a method of achieving hierarchical clustering of a data set with the GSOM. Such hierarchical clustering allows the data analyst to identify significant and interesting clusters at a higher level of the hierarchy, and as such continue with finer clustering of only the interesting clusters. Therefore, only a small map is created in the beginning with a low spread factor, which can be generated for even a very large data set. Further analysis is conducted on selected sections of the data and as such of smaller volume. Therefore, this method facilitates the analysis of even very large data sets.

Growing self-organizing map (GSOM) has been characterized as a knowledge discovery visualization application which outshines the traditional self-organizing map (SOM) due to its dynamic structure in which nodes can grow based on the input data. GSOM is utilized as a visualization tool in this paper to cluster fMRI finger tapping and non- tapping data, demonstrating the visualization capability to distinguish between tapping or non-tapping. A unique feature of GSOM is a parameter called the spread factor whose functionality is to control the spread of the GSOM map. By setting different levels of spread factor, different granularities of region of interests within tapping or non-tapping images can be visualized and analyzed. Euclidean distance based similarity calculation is used to quantify the visualized difference between tapping and non tapping images. Once the differences are identified, the spread factor is used to generate a more detailed view of those regions to provide a better visualization of the brain regions.

Self Organizing Maps (SOM) are widely used in data mining and high-dimensional data visualization due to its unsupervised nature and robustness. Growing Self Organizing Maps (GSOM) is a variant of SOM algorithm which allows nodes to be grown so that it can represent the input space better. Without using a fixed 2D grid like SOM, GSOM starts with four nodes and keeps track of the quantization error in each node. New nodes are grown from an existing node if its error value exceeds a pre-defined threshold. Ability of the GSOM algorithm to represent input space accurately is vital to extend its applicability to a wider spectrum of problems. This ability can be improved by identifying nodes that represent low probability regions in the input space and removing them periodically from the map. This will improve the homogeneity and completeness of the final clustering result. A new extension to GSOM algorithm based on node deletion is proposed in this paper as a solution to this problem. Furthermore, two new algorithms inspired by cache replacement policies are presented. First algorithm is based on Adaptive Replacement Cache (ARC) and maintains two separate Least Recently Used (LRU) lists of the nodes. Second algorithm is built on Frequency Based Replacement policy (FBR) and maintains a single LRU list. These algorithms consider both recent and frequent trends in the GSOM grid before deciding on the nodes to be deleted. The experiments conducted suggest that the FBR based method for node deletion outperforms the standard algorithm and other existing node deletion methods.

The Growing Self Organising Map (GSOM) has been presented as an extended version of the Self Organising Map (SOM) which has significant advantages for knowledge discovery applications. In this article, we present a further extension to the GSOM in which the cluster identification process can be automated. The self-generating ability of the GSOM is used to identify the paths along which the GSOM grew, and these paths are used to develop a skeleton of the data set. Such a skeleton is then used as a base for separating the clusters in the data

This paper presents a multi agent system which provides a method for cooperation and coordination among agents using collective reasoning. There are two types of agents in the system: distributed agents and a central administrator agent. The distributed agents provide local knowledge extraction tasks and communicate it to the central administrator agent. The central administrator agent builds a global picture about the problem domain using the local knowledge of each agent. It uses the Growing Self Organizing Map (GSOM), which is a dynamic version of the Self Organizing Map (SOM) for analyzing the global knowledge. The system differs from similar systems which employ central administrator functionality as the central administrator agent in the proposed system is capable of autonomously refining the task model of the distributed agents and dynamically determining the efficient number of agents required in the system.

The clustering of high-dimensional data presents a critical computational problem. Therefore, it is convenient to arrange the cluster centres on a grid with a small dimensional space that reduces computational cost and can be easily visualized. Moreover, in real applications there is no sharp boundary between classes, real datasets are naturally defined in a fuzzy context. Therefore, fuzzy clustering fits better for complex real datasets to determine the best distribution. Self-organizing map (SOM) technique is appropriate for clustering and vector quantization of high-dimensional data. In this paper we present a new fuzzy learning approach called FMIG (fuzzy multilevel interior growing self-organizing maps). The proposed algorithm is a fuzzy version of MIGSOM (multilevel interior growing self-organizing maps). The main contribution of FMIG is to define a fuzzy process of data mapping and to take into account the fuzzy aspect of high-dimensional real datasets. This new algorithm is able to auto-organize the map accordingly to the fuzzy training property of the nodes. In the second step, the introduced scheme for FMIG is clustered by means of fuzzy C-means (FCM) to discover the interior class distribution of the learned data. The validation of FCM partitions is directed by applying six validity indexes. Superiority of the new method is demonstrated by comparing it with crisp MIGSOM, GSOM (growing SOM) and FKCN (fuzzy Kohonen clustering network) techniques. Thus, our new method shows improvement in term of quantization error, topology preservation and clustering ability.

Metagenomic projects using whole-genome shotgun (WGS) sequencing produces many unassembled DNA sequences and small contigs. The step of clustering these sequences, based on biological and molecular features, is called binning. A reported strategy for binning that combines oligonucleotide frequency and self-organising maps (SOM) shows high potential. We improve this strategy by identifying suitable training features, implementing a better clustering algorithm, and defining quantitative measures for assessing results. We investigated the suitability of each of di-, tri-, tetra-, and pentanucleotide frequencies. The results show that dinucleotide frequency is not a sufficiently strong signature for binning 10 kb long DNA sequences, compared to the other three. Furthermore, we observed that increased order of oligonucleotide frequency may deteriorate the assignment result in some cases, which indicates the possible existence of optimal species-specific oligonucleotide frequency. We replaced SOM with growing self-organising map (GSOM) where comparable results are obtained while gaining speed improvement.

This article, written by Special Publications Editor Adam Wilson, contains highlights of paper SPE 190087, “Unsupervised Statistical Learning With Integrated Pattern-Based Geostatistical Simulation,” by Q. Li and R. Aguilera, SPE, University of Calgary, prepared for the 2018 SPE Western Regional Meeting, Garden Grove, California, USA, 22–27 April. The paper has not been peer reviewed. This paper presents a new geostatistics modeling methodology that connects geostatistics and machine-learning methodologies, uses nonlinear topological mapping to reduce the original high-dimensional data space, and uses unsupervised-learning algorithms to bypass problems with supervised-learning algorithms. The algorithm presented is a neural topology-preserving pattern-based geostatistical simulation algorithm that integrates the self-organizing map (SOM) concept and its updated version—growing self-organizing map (GSOM)—with an unsupervised competitive learning structure. Introduction In oil and gas reservoir modeling, any model construction faces challenges of limited data to some extent. The heuristic behind all geostatistical techniques is the implicit existence of statistical relationships among available data. “Data” here is a broad term; it could be discrete points, such as porosity or permeability at certain locations, but it also could be training images (TIs), which are used in this work. Using TIs as input data originated with multiple-point geostatistics. The aim was to overcome the limitations of using traditional two-point statistical variograms to describe geological continuity, especially in the case of curvilinear structures, which are quite common in nature, such as in fracture networks and geological fluvial structures. The authors write that geostatistics could benefit from the machine-learning or statistical-learning areas. Machine-learning tasks can be divided into two protocols, supervised learning and unsupervised learning. The difference depends on whether the input data have correct labels or not. For the investigation considered in this paper, after retrieving image patches from TIs, machine-learning algorithms were used to cluster those image patches into different classifications. If the correct clusters to which the image patches belong is known beforehand, the data can be said to have correct labels. Further, a large amount of these labeled data could be used for a model to learn. Supervised learning occurs when the model is finely tuned by guidance of these correct labels using an error-correction process. Unsupervised learning, on the other hand, occurs when neither how many clusters exist nor the correct clusters to which the image patches belong is known. Thus, without correct labels for guidance, this problem is identified as an unsupervised-learning problem. This is the first major drawback of applying machine-learning algorithms to geostatistical simulations. Two other important issues are encountered when performing pattern-based geostatistical simulations. The first is related to the large number of image patches. This large image-patch database contains high pattern redundancy, which could make pattern similarity comparison inefficient. The second issue is the high dimensionality of each image patch, which could typically be described by low-dimensional nonlinear-structure manifolds. To model and visualize these high-dimensional data, a nonlinearity dimensionality reduction should be sought.

The manufacturing process is complex due to the large number of processes, diverse equipment set and nonlinear process flows. Manufacturers constantly face yield and quality problems as they constantly redesign their processes for the rapid introduction of new products and adoption of new process technologies. Solving product yield and quality problems in a manufacturing process is becoming increasingly difficult. There are various types of failures and their causes have complex multi-factor interrelationships. High innovation speed forced today's manufacturers to find failure causes quickly by examining the historical manufacturing data. Data mining offers tools for quick discovery of relationships, patterns, and knowledge in large databases. This has been applied to many fields such as biological technology, financial analysis, medical information, etc. Application of data mining to manufacturing is relatively limited mainly because of complexity of manufacturing data. Growing self-organizing map (GSOM) algorithm has been proven to be an efficient algorithm to analyze unsupervised DNA data. However, it produced unsatisfactory clustering when used on some manufacturing data. Moreover, there was no benchmark to monitor improvement in clustering. In this study a method has been proposed to evaluate quality of the clusters produced by GSOM and to remove insignificant variables from the dataset. With the proposed modifications, significant improvement in unsupervised clustering was achieved with complex manufacturing data. Results show that the proposed method is able to effectively differentiate good and faulty products.

Inference of hidden common driver dynamics by anisotropic self-organizing neural networks.

In the traditional self-organizing map (SOM) the best matching unit (BMU) affects other neurons, through the learning rule, as a function of distance. Here, we propose a new parameter in the learning rule so neurons are not only affected by BMU as a function of distance, but as a function of the frequency of activation from both, the BMU and input vectors, to the affected neurons. This frequency parameter allows non radial neighborhoods and the quality of the formed maps is improved with respect to those formed by traditional SOM, as we show by comparing several error measures and five data sets.KeywordsInput VectorLearning RuleError QuantizationFrequency FunctionLearning FactorThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This paper proposes a novel approach for a Constructive Self-Organizing Map (SOM) based world modeling for search and rescue operations in disaster environments. In our approach, nodes of the self organizing network consist of victim and waypoint classes where victim denotes a human being waiting to be rescued and waypoint denotes a free space that can be reached from the entrance of debris. The proposed approach performed better than traditional self-organizing maps in terms of both the accuracy of the output and the learning speed. In this paper the detailed explanation of the approach and some experimental results are given.

Class structure visualization with semi-supervised growing self-organizing maps

Solving product yield and quality problems in a manufacturing process is becoming increasingly more difficult. There are various types of failures and their causes have complex multi-factor interrelationships. Semiconductor manufacturing is very complex due to the large number of processes, diverse equipment set and nonlinear process flows. Its manufacturing database comprises of hundreds of process control, process step and wafer probe data. This huge volume of data coupled with quicker time to market expectations is making finding and resolving problems quickly an overwhelming task. In this study, a methodology developed using dynamic growing self-organizing map (GSOM) to detect the faulty products in a wafer manufacturing process. As part of the methodology, a clustering quality measure was developed to evaluate the performance of the algorithm in separating good and faulty products. Results show that the algorithm was able to separate good and faulty products from the raw data. Even though this work has focused mainly on clustering good and faulty products, the technique can be extended to model the failure causes of the lower yielding products.