Dynamic self-organizing maps with controlled growth for knowledge discovery

We have been working on the application of Machining Learning in Metabolomics, Drug-Drug Interaction Discovery and Human Gait Recognition [1-5], profiling large data sets. Extraction of vital information about 1) plant metabolomics that can improve the environment and food quality, 2) in vitro neuronal network behavior patterns for various drugs that can be used to characterize drugs for brain deceases and 3) human gait patterns that can reveal various diseases were among these applications. We have demonstrated with considerable success in using unsupervised clustering techniques to analyze genetic and metabolomic data. This includes analysis of drought resistance in wheat [4] and microbial metagenomes [5].We introduced two methods: Near Unsupervised Learning (NUL) and Sub-sample Error Graphs (SEGs) [5] to analyze large amount of data. Self Organizing Map (SOM), which is one of the widely used Unsupervised Neural Networks, has been used as a data-mining tool due to its ability to map high dimensional data into a two dimensional feature map, which is expected to be topology preserving allowing users to visually identify clusters by their topological relationships in terms of their proximity on the map. The Growing Self Organizing Maps (GSOM) further allows the map size to be determined by the algorithm, which relies upon a user set parameter called Spread Factor (SF) [6]. The wide availability of GPUs for affordable prices allows faster comparison of various SOMs with different maps sizes taking away some of the advantages GSOM claimed. Further development of GSOM into a Dynamic SOM Tree exploits the possibility of varying SF to obtain multiple GSOMs from a small number of compact clusters to a large number of sparse clusters [7]. NUL methods can be applied on GSOM using a small number of labels that should be available for every class. This is however not realistic in some applications where the number of classes cannot be explicitly known. We propose a new method call Deep Near Unsupervised Learning (D-NUL), where Dynamic SOM tree is used instead of GSOM and the number of classes are not assumed to be known. The implementation of Dynamic SOM tree methods with varying SF on GPUs will make the computation with D-NUL possible for many problems in big data analytics.

The growing self-organising map (GSOM) has recently been proposed as an alternative neural network architecture based on the traditional self-organising map (SOM). The GSOM provides the user with the ability to control the spread of the map by defining a parameter called the spread factor (SF), which results in enhanced data mining and hierarchical clustering opportunities. When experimenting with the SOM, the grid size (number of rows and columns of nodes) can be changed until a suitable cluster distribution is achieved. In this paper we highlight the effect of the spread factor on the GSOM and contrast this effect with grid size change (increase and decrease) in the SOM. We also present experimental results in support of our claims regarding differences between GSOM and SOM.

Growing self-organizing map (GSOM) has been characterized as a knowledge discovery visualization application which outshines the traditional self-organizing map (SOM) due to its dynamic structure in which nodes can grow based on the input data. GSOM is utilized as a visualization tool in this paper to cluster fMRI finger tapping and non- tapping data, demonstrating the visualization capability to distinguish between tapping or non-tapping. A unique feature of GSOM is a parameter called the spread factor whose functionality is to control the spread of the GSOM map. By setting different levels of spread factor, different granularities of region of interests within tapping or non-tapping images can be visualized and analyzed. Euclidean distance based similarity calculation is used to quantify the visualized difference between tapping and non tapping images. Once the differences are identified, the spread factor is used to generate a more detailed view of those regions to provide a better visualization of the brain regions.

Self Organizing Maps (SOM) are widely used in data mining and high-dimensional data visualization due to its unsupervised nature and robustness. Growing Self Organizing Maps (GSOM) is a variant of SOM algorithm which allows nodes to be grown so that it can represent the input space better. Without using a fixed 2D grid like SOM, GSOM starts with four nodes and keeps track of the quantization error in each node. New nodes are grown from an existing node if its error value exceeds a pre-defined threshold. Ability of the GSOM algorithm to represent input space accurately is vital to extend its applicability to a wider spectrum of problems. This ability can be improved by identifying nodes that represent low probability regions in the input space and removing them periodically from the map. This will improve the homogeneity and completeness of the final clustering result. A new extension to GSOM algorithm based on node deletion is proposed in this paper as a solution to this problem. Furthermore, two new algorithms inspired by cache replacement policies are presented. First algorithm is based on Adaptive Replacement Cache (ARC) and maintains two separate Least Recently Used (LRU) lists of the nodes. Second algorithm is built on Frequency Based Replacement policy (FBR) and maintains a single LRU list. These algorithms consider both recent and frequent trends in the GSOM grid before deciding on the nodes to be deleted. The experiments conducted suggest that the FBR based method for node deletion outperforms the standard algorithm and other existing node deletion methods.

The Growing Self Organising Map (GSOM) has been presented as an extended version of the Self Organising Map (SOM) which has significant advantages for knowledge discovery applications. In this article, we present a further extension to the GSOM in which the cluster identification process can be automated. The self-generating ability of the GSOM is used to identify the paths along which the GSOM grew, and these paths are used to develop a skeleton of the data set. Such a skeleton is then used as a base for separating the clusters in the data

This paper presents a multi agent system which provides a method for cooperation and coordination among agents using collective reasoning. There are two types of agents in the system: distributed agents and a central administrator agent. The distributed agents provide local knowledge extraction tasks and communicate it to the central administrator agent. The central administrator agent builds a global picture about the problem domain using the local knowledge of each agent. It uses the Growing Self Organizing Map (GSOM), which is a dynamic version of the Self Organizing Map (SOM) for analyzing the global knowledge. The system differs from similar systems which employ central administrator functionality as the central administrator agent in the proposed system is capable of autonomously refining the task model of the distributed agents and dynamically determining the efficient number of agents required in the system.

The clustering of high-dimensional data presents a critical computational problem. Therefore, it is convenient to arrange the cluster centres on a grid with a small dimensional space that reduces computational cost and can be easily visualized. Moreover, in real applications there is no sharp boundary between classes, real datasets are naturally defined in a fuzzy context. Therefore, fuzzy clustering fits better for complex real datasets to determine the best distribution. Self-organizing map (SOM) technique is appropriate for clustering and vector quantization of high-dimensional data. In this paper we present a new fuzzy learning approach called FMIG (fuzzy multilevel interior growing self-organizing maps). The proposed algorithm is a fuzzy version of MIGSOM (multilevel interior growing self-organizing maps). The main contribution of FMIG is to define a fuzzy process of data mapping and to take into account the fuzzy aspect of high-dimensional real datasets. This new algorithm is able to auto-organize the map accordingly to the fuzzy training property of the nodes. In the second step, the introduced scheme for FMIG is clustered by means of fuzzy C-means (FCM) to discover the interior class distribution of the learned data. The validation of FCM partitions is directed by applying six validity indexes. Superiority of the new method is demonstrated by comparing it with crisp MIGSOM, GSOM (growing SOM) and FKCN (fuzzy Kohonen clustering network) techniques. Thus, our new method shows improvement in term of quantization error, topology preservation and clustering ability.

The manufacturing process is complex due to the large number of processes, diverse equipment set and nonlinear process flows. Manufacturers constantly face yield and quality problems as they constantly redesign their processes for the rapid introduction of new products and adoption of new process technologies. Solving product yield and quality problems in a manufacturing process is becoming increasingly difficult. There are various types of failures and their causes have complex multi-factor interrelationships. High innovation speed forced today's manufacturers to find failure causes quickly by examining the historical manufacturing data. Data mining offers tools for quick discovery of relationships, patterns, and knowledge in large databases. This has been applied to many fields such as biological technology, financial analysis, medical information, etc. Application of data mining to manufacturing is relatively limited mainly because of complexity of manufacturing data. Growing self-organizing map (GSOM) algorithm has been proven to be an efficient algorithm to analyze unsupervised DNA data. However, it produced unsatisfactory clustering when used on some manufacturing data. Moreover, there was no benchmark to monitor improvement in clustering. In this study a method has been proposed to evaluate quality of the clusters produced by GSOM and to remove insignificant variables from the dataset. With the proposed modifications, significant improvement in unsupervised clustering was achieved with complex manufacturing data. Results show that the proposed method is able to effectively differentiate good and faulty products.

Metagenomic projects using whole-genome shotgun (WGS) sequencing produces many unassembled DNA sequences and small contigs. The step of clustering these sequences, based on biological and molecular features, is called binning. A reported strategy for binning that combines oligonucleotide frequency and self-organising maps (SOM) shows high potential. We improve this strategy by identifying suitable training features, implementing a better clustering algorithm, and defining quantitative measures for assessing results. We investigated the suitability of each of di-, tri-, tetra-, and pentanucleotide frequencies. The results show that dinucleotide frequency is not a sufficiently strong signature for binning 10 kb long DNA sequences, compared to the other three. Furthermore, we observed that increased order of oligonucleotide frequency may deteriorate the assignment result in some cases, which indicates the possible existence of optimal species-specific oligonucleotide frequency. We replaced SOM with growing self-organising map (GSOM) where comparable results are obtained while gaining speed improvement.

This article, written by Special Publications Editor Adam Wilson, contains highlights of paper SPE 190087, “Unsupervised Statistical Learning With Integrated Pattern-Based Geostatistical Simulation,” by Q. Li and R. Aguilera, SPE, University of Calgary, prepared for the 2018 SPE Western Regional Meeting, Garden Grove, California, USA, 22–27 April. The paper has not been peer reviewed. This paper presents a new geostatistics modeling methodology that connects geostatistics and machine-learning methodologies, uses nonlinear topological mapping to reduce the original high-dimensional data space, and uses unsupervised-learning algorithms to bypass problems with supervised-learning algorithms. The algorithm presented is a neural topology-preserving pattern-based geostatistical simulation algorithm that integrates the self-organizing map (SOM) concept and its updated version—growing self-organizing map (GSOM)—with an unsupervised competitive learning structure. Introduction In oil and gas reservoir modeling, any model construction faces challenges of limited data to some extent. The heuristic behind all geostatistical techniques is the implicit existence of statistical relationships among available data. “Data” here is a broad term; it could be discrete points, such as porosity or permeability at certain locations, but it also could be training images (TIs), which are used in this work. Using TIs as input data originated with multiple-point geostatistics. The aim was to overcome the limitations of using traditional two-point statistical variograms to describe geological continuity, especially in the case of curvilinear structures, which are quite common in nature, such as in fracture networks and geological fluvial structures. The authors write that geostatistics could benefit from the machine-learning or statistical-learning areas. Machine-learning tasks can be divided into two protocols, supervised learning and unsupervised learning. The difference depends on whether the input data have correct labels or not. For the investigation considered in this paper, after retrieving image patches from TIs, machine-learning algorithms were used to cluster those image patches into different classifications. If the correct clusters to which the image patches belong is known beforehand, the data can be said to have correct labels. Further, a large amount of these labeled data could be used for a model to learn. Supervised learning occurs when the model is finely tuned by guidance of these correct labels using an error-correction process. Unsupervised learning, on the other hand, occurs when neither how many clusters exist nor the correct clusters to which the image patches belong is known. Thus, without correct labels for guidance, this problem is identified as an unsupervised-learning problem. This is the first major drawback of applying machine-learning algorithms to geostatistical simulations. Two other important issues are encountered when performing pattern-based geostatistical simulations. The first is related to the large number of image patches. This large image-patch database contains high pattern redundancy, which could make pattern similarity comparison inefficient. The second issue is the high dimensionality of each image patch, which could typically be described by low-dimensional nonlinear-structure manifolds. To model and visualize these high-dimensional data, a nonlinearity dimensionality reduction should be sought.

High quality ground truth data sets are crucial for the development of image recognition systems. However, the task of annotating large image data sets manually takes a lot of time and effort. In order to lower the burden for the development of application-specific image recognition systems, we developed an advanced user interface. This interface is especially designed for non-expert users with little-to-no knowledge of computer vision techniques. The interface presents images clustered by similarity and allows for an efficient and simple annotation of large data sets. The integration of overview+detail concepts allows the precise navigation inside large data sets. The interface can be used without prior instructions on the underlying concepts, like self-organizing maps, image features or visualization techniques.

In general, visual clusters are preferred over large data sets; this is an attempt to take advantage of cluster techniques to reduce the mathematical complexity of small data sets. To identify the possibility of implementing the clustering technique in a small dataset, the wear observations of PLA/Cu composite samples printed using the Fused Deposition Model (FDM) is taken into consideration. In this study, the Self Organizing Map (SOM) tool as a non-supervised Neural Network (NN) is used to visualize the data. Here, SOM combinations with vector quantification and projection are used to identify or rank the wear machinability parameters on the new composite filament printed under different FDM conditions. The competitive layer in SOM will classify the given parameters of the wear machine (vectors) at any number of dimensions may be into several groups of layer neurons. The limitation of SOM is map size which cannot exceed 1000 units of training. However, for the small data set under consideration, the extent of these limits will not affect performance. The SOM algorithm developed for the study of wear provides the outlet within the acceptable range. In addition, the linear regression analysis is carried out for the output response to measure the wear characteristics of the machining observation.

Data mining or knowledge discovery in database (KDD) refers to the non-trivial process of discovering interesting, implicit, and previously unknown knowledge from databases.Such a task implies to be able to perform analyses both on high-dimensional input data and large dataset.The most popular models used in KDD are the symbolic models.Unfortunately, these models suffer of very serious limitations.Rule generation is a highly time-consuming process that generates a huge number of rules, including a large ratio of redundant rules.Hence, this prohibits any kind of rule computation and selection as soon as data are numerous and they are represented by very high-dimensional description space.This latter situation is very often encountered with documentary data.To cope with these problems, preliminary KDD trials using numerical models have been made.An algorithm for knowledge extraction from self-organizing network has been proposed in [8].This approach is based on a supervised generalized relevance learning vector quantization (GRLVQ) which is used for extracting decision trees.The different paths of the generated trees are then used for denoting rules.Nevertheless, the main defect of this method is to necessitate training data.On our own side, we have proposed a hybrid classification method for mapping an explicative structure issued from a symbolic classification into an unsupervised numerical self-organizing map (SOM) [15].SOM map and Galois lattice are generated on the same data.The cosine projection is then used for associating lattice concepts to the SOM classes.Concepts properties act as explanation for the SOM classes.Furthermore, lattice pruning combined with migration of the associated SOM classes towards the top of the pruned lattice is used to generate explanation of increasing scope on the SOM map.Association rules can also be produced in such a way.Although it establishes interesting links between numerical and symbolic worlds this approach necessitates the time-consuming computation of a whole Galois lattice.In a parallel way, in order to enhance both the quality and the granularity of the data analysis and to reduce the noise which is inevitably generated in an overall classification approach, we have introduced the multi-viewpoint analysis and multilevel clustering approach based on a significant extension of the SOM model, named MultiSOM [19][25].The viewpoint building principle consists in separating the description of the data into several sub-descriptions corresponding different property subsets or even different data subsets.In MultiSOM each viewpoint is represented by a single SOM map.www.intechopen.com

Class structure visualization with semi-supervised growing self-organizing maps

Is power everything? What can we learn from large data sets