Mining a Growing Feature Map by Data Skeleton Modelling

We have been working on the application of Machining Learning in Metabolomics, Drug-Drug Interaction Discovery and Human Gait Recognition [1-5], profiling large data sets. Extraction of vital information about 1) plant metabolomics that can improve the environment and food quality, 2) in vitro neuronal network behavior patterns for various drugs that can be used to characterize drugs for brain deceases and 3) human gait patterns that can reveal various diseases were among these applications. We have demonstrated with considerable success in using unsupervised clustering techniques to analyze genetic and metabolomic data. This includes analysis of drought resistance in wheat [4] and microbial metagenomes [5].We introduced two methods: Near Unsupervised Learning (NUL) and Sub-sample Error Graphs (SEGs) [5] to analyze large amount of data. Self Organizing Map (SOM), which is one of the widely used Unsupervised Neural Networks, has been used as a data-mining tool due to its ability to map high dimensional data into a two dimensional feature map, which is expected to be topology preserving allowing users to visually identify clusters by their topological relationships in terms of their proximity on the map. The Growing Self Organizing Maps (GSOM) further allows the map size to be determined by the algorithm, which relies upon a user set parameter called Spread Factor (SF) [6]. The wide availability of GPUs for affordable prices allows faster comparison of various SOMs with different maps sizes taking away some of the advantages GSOM claimed. Further development of GSOM into a Dynamic SOM Tree exploits the possibility of varying SF to obtain multiple GSOMs from a small number of compact clusters to a large number of sparse clusters [7]. NUL methods can be applied on GSOM using a small number of labels that should be available for every class. This is however not realistic in some applications where the number of classes cannot be explicitly known. We propose a new method call Deep Near Unsupervised Learning (D-NUL), where Dynamic SOM tree is used instead of GSOM and the number of classes are not assumed to be known. The implementation of Dynamic SOM tree methods with varying SF on GPUs will make the computation with D-NUL possible for many problems in big data analytics.

Self Organizing Maps (SOM) are widely used in data mining and high-dimensional data visualization due to its unsupervised nature and robustness. Growing Self Organizing Maps (GSOM) is a variant of SOM algorithm which allows nodes to be grown so that it can represent the input space better. Without using a fixed 2D grid like SOM, GSOM starts with four nodes and keeps track of the quantization error in each node. New nodes are grown from an existing node if its error value exceeds a pre-defined threshold. Ability of the GSOM algorithm to represent input space accurately is vital to extend its applicability to a wider spectrum of problems. This ability can be improved by identifying nodes that represent low probability regions in the input space and removing them periodically from the map. This will improve the homogeneity and completeness of the final clustering result. A new extension to GSOM algorithm based on node deletion is proposed in this paper as a solution to this problem. Furthermore, two new algorithms inspired by cache replacement policies are presented. First algorithm is based on Adaptive Replacement Cache (ARC) and maintains two separate Least Recently Used (LRU) lists of the nodes. Second algorithm is built on Frequency Based Replacement policy (FBR) and maintains a single LRU list. These algorithms consider both recent and frequent trends in the GSOM grid before deciding on the nodes to be deleted. The experiments conducted suggest that the FBR based method for node deletion outperforms the standard algorithm and other existing node deletion methods.

The growing self-organising map (GSOM) has recently been proposed as an alternative neural network architecture based on the traditional self-organising map (SOM). The GSOM provides the user with the ability to control the spread of the map by defining a parameter called the spread factor (SF), which results in enhanced data mining and hierarchical clustering opportunities. When experimenting with the SOM, the grid size (number of rows and columns of nodes) can be changed until a suitable cluster distribution is achieved. In this paper we highlight the effect of the spread factor on the GSOM and contrast this effect with grid size change (increase and decrease) in the SOM. We also present experimental results in support of our claims regarding differences between GSOM and SOM.

The growing self-organizing map (GSOM) has been presented as an extended version of the self-organizing map (SOM), which has significant advantages for knowledge discovery applications. In this paper, the GSOM algorithm is presented in detail and the effect of a spread factor, which can be used to measure and control the spread of the GSOM, is investigated. The spread factor is independent of the dimensionality of the data and as such can be used as a controlling measure for generating maps with different dimensionality, which can then be compared and analyzed with better accuracy. The spread factor is also presented as a method of achieving hierarchical clustering of a data set with the GSOM. Such hierarchical clustering allows the data analyst to identify significant and interesting clusters at a higher level of the hierarchy, and as such continue with finer clustering of only the interesting clusters. Therefore, only a small map is created in the beginning with a low spread factor, which can be generated for even a very large data set. Further analysis is conducted on selected sections of the data and as such of smaller volume. Therefore, this method facilitates the analysis of even very large data sets.

Growing self-organizing map (GSOM) has been characterized as a knowledge discovery visualization application which outshines the traditional self-organizing map (SOM) due to its dynamic structure in which nodes can grow based on the input data. GSOM is utilized as a visualization tool in this paper to cluster fMRI finger tapping and non- tapping data, demonstrating the visualization capability to distinguish between tapping or non-tapping. A unique feature of GSOM is a parameter called the spread factor whose functionality is to control the spread of the GSOM map. By setting different levels of spread factor, different granularities of region of interests within tapping or non-tapping images can be visualized and analyzed. Euclidean distance based similarity calculation is used to quantify the visualized difference between tapping and non tapping images. Once the differences are identified, the spread factor is used to generate a more detailed view of those regions to provide a better visualization of the brain regions.

This paper presents a multi agent system which provides a method for cooperation and coordination among agents using collective reasoning. There are two types of agents in the system: distributed agents and a central administrator agent. The distributed agents provide local knowledge extraction tasks and communicate it to the central administrator agent. The central administrator agent builds a global picture about the problem domain using the local knowledge of each agent. It uses the Growing Self Organizing Map (GSOM), which is a dynamic version of the Self Organizing Map (SOM) for analyzing the global knowledge. The system differs from similar systems which employ central administrator functionality as the central administrator agent in the proposed system is capable of autonomously refining the task model of the distributed agents and dynamically determining the efficient number of agents required in the system.

The clustering of high-dimensional data presents a critical computational problem. Therefore, it is convenient to arrange the cluster centres on a grid with a small dimensional space that reduces computational cost and can be easily visualized. Moreover, in real applications there is no sharp boundary between classes, real datasets are naturally defined in a fuzzy context. Therefore, fuzzy clustering fits better for complex real datasets to determine the best distribution. Self-organizing map (SOM) technique is appropriate for clustering and vector quantization of high-dimensional data. In this paper we present a new fuzzy learning approach called FMIG (fuzzy multilevel interior growing self-organizing maps). The proposed algorithm is a fuzzy version of MIGSOM (multilevel interior growing self-organizing maps). The main contribution of FMIG is to define a fuzzy process of data mapping and to take into account the fuzzy aspect of high-dimensional real datasets. This new algorithm is able to auto-organize the map accordingly to the fuzzy training property of the nodes. In the second step, the introduced scheme for FMIG is clustered by means of fuzzy C-means (FCM) to discover the interior class distribution of the learned data. The validation of FCM partitions is directed by applying six validity indexes. Superiority of the new method is demonstrated by comparing it with crisp MIGSOM, GSOM (growing SOM) and FKCN (fuzzy Kohonen clustering network) techniques. Thus, our new method shows improvement in term of quantization error, topology preservation and clustering ability.

The manufacturing process is complex due to the large number of processes, diverse equipment set and nonlinear process flows. Manufacturers constantly face yield and quality problems as they constantly redesign their processes for the rapid introduction of new products and adoption of new process technologies. Solving product yield and quality problems in a manufacturing process is becoming increasingly difficult. There are various types of failures and their causes have complex multi-factor interrelationships. High innovation speed forced today's manufacturers to find failure causes quickly by examining the historical manufacturing data. Data mining offers tools for quick discovery of relationships, patterns, and knowledge in large databases. This has been applied to many fields such as biological technology, financial analysis, medical information, etc. Application of data mining to manufacturing is relatively limited mainly because of complexity of manufacturing data. Growing self-organizing map (GSOM) algorithm has been proven to be an efficient algorithm to analyze unsupervised DNA data. However, it produced unsatisfactory clustering when used on some manufacturing data. Moreover, there was no benchmark to monitor improvement in clustering. In this study a method has been proposed to evaluate quality of the clusters produced by GSOM and to remove insignificant variables from the dataset. With the proposed modifications, significant improvement in unsupervised clustering was achieved with complex manufacturing data. Results show that the proposed method is able to effectively differentiate good and faulty products.

Metagenomic projects using whole-genome shotgun (WGS) sequencing produces many unassembled DNA sequences and small contigs. The step of clustering these sequences, based on biological and molecular features, is called binning. A reported strategy for binning that combines oligonucleotide frequency and self-organising maps (SOM) shows high potential. We improve this strategy by identifying suitable training features, implementing a better clustering algorithm, and defining quantitative measures for assessing results. We investigated the suitability of each of di-, tri-, tetra-, and pentanucleotide frequencies. The results show that dinucleotide frequency is not a sufficiently strong signature for binning 10 kb long DNA sequences, compared to the other three. Furthermore, we observed that increased order of oligonucleotide frequency may deteriorate the assignment result in some cases, which indicates the possible existence of optimal species-specific oligonucleotide frequency. We replaced SOM with growing self-organising map (GSOM) where comparable results are obtained while gaining speed improvement.

This article, written by Special Publications Editor Adam Wilson, contains highlights of paper SPE 190087, “Unsupervised Statistical Learning With Integrated Pattern-Based Geostatistical Simulation,” by Q. Li and R. Aguilera, SPE, University of Calgary, prepared for the 2018 SPE Western Regional Meeting, Garden Grove, California, USA, 22–27 April. The paper has not been peer reviewed. This paper presents a new geostatistics modeling methodology that connects geostatistics and machine-learning methodologies, uses nonlinear topological mapping to reduce the original high-dimensional data space, and uses unsupervised-learning algorithms to bypass problems with supervised-learning algorithms. The algorithm presented is a neural topology-preserving pattern-based geostatistical simulation algorithm that integrates the self-organizing map (SOM) concept and its updated version—growing self-organizing map (GSOM)—with an unsupervised competitive learning structure. Introduction In oil and gas reservoir modeling, any model construction faces challenges of limited data to some extent. The heuristic behind all geostatistical techniques is the implicit existence of statistical relationships among available data. “Data” here is a broad term; it could be discrete points, such as porosity or permeability at certain locations, but it also could be training images (TIs), which are used in this work. Using TIs as input data originated with multiple-point geostatistics. The aim was to overcome the limitations of using traditional two-point statistical variograms to describe geological continuity, especially in the case of curvilinear structures, which are quite common in nature, such as in fracture networks and geological fluvial structures. The authors write that geostatistics could benefit from the machine-learning or statistical-learning areas. Machine-learning tasks can be divided into two protocols, supervised learning and unsupervised learning. The difference depends on whether the input data have correct labels or not. For the investigation considered in this paper, after retrieving image patches from TIs, machine-learning algorithms were used to cluster those image patches into different classifications. If the correct clusters to which the image patches belong is known beforehand, the data can be said to have correct labels. Further, a large amount of these labeled data could be used for a model to learn. Supervised learning occurs when the model is finely tuned by guidance of these correct labels using an error-correction process. Unsupervised learning, on the other hand, occurs when neither how many clusters exist nor the correct clusters to which the image patches belong is known. Thus, without correct labels for guidance, this problem is identified as an unsupervised-learning problem. This is the first major drawback of applying machine-learning algorithms to geostatistical simulations. Two other important issues are encountered when performing pattern-based geostatistical simulations. The first is related to the large number of image patches. This large image-patch database contains high pattern redundancy, which could make pattern similarity comparison inefficient. The second issue is the high dimensionality of each image patch, which could typically be described by low-dimensional nonlinear-structure manifolds. To model and visualize these high-dimensional data, a nonlinearity dimensionality reduction should be sought.

Class structure visualization with semi-supervised growing self-organizing maps

Solving product yield and quality problems in a manufacturing process is becoming increasingly more difficult. There are various types of failures and their causes have complex multi-factor interrelationships. Semiconductor manufacturing is very complex due to the large number of processes, diverse equipment set and nonlinear process flows. Its manufacturing database comprises of hundreds of process control, process step and wafer probe data. This huge volume of data coupled with quicker time to market expectations is making finding and resolving problems quickly an overwhelming task. In this study, a methodology developed using dynamic growing self-organizing map (GSOM) to detect the faulty products in a wafer manufacturing process. As part of the methodology, a clustering quality measure was developed to evaluate the performance of the algorithm in separating good and faulty products. Results show that the algorithm was able to separate good and faulty products from the raw data. Even though this work has focused mainly on clustering good and faulty products, the technique can be extended to model the failure causes of the lower yielding products.

With an onset of social network amount of data is increasing day by day. In order to analyze the data and extract useful information, the data mining technology can be used. Clustering is one of the popular method of data mining. Clustering can be used for visualizing and analyzing of data. We are discussing Kohonen SOM. We are using neural networks, as a data mining tool which provides statistical observation and layout from big data-sets. We determine how Self-Organizing Kohonen Maps, can be efficiently used for data mining purposes. The Self Organizing Map (SOM) unsupervised learning is an effective computational tool in data mining processes. Self-Organizing Maps (SOMs) used to visualize social network dataset. We used Self-Organizing Map for clustering and analyzing high-dimensional and complex social network datasets. This paper also visualizes SOM neighbor connection, SOM neighbor weight distance, SOM weight position. We perform self organizing map algorithm for social network dataset in matlab.

Summary Well-test data have been used traditionally for determining a variety of reservoir parameters, such as average permeability, storage capacity, reservoir damage, presence of faults and fractures, and reservoir mechanism. A number of techniques, both conventional methods, such as type-curve matching and numerical simulation, and artificial intelligence (AI) methods, have been used for identifying well-test models. These methods are laborious and time-consuming and at times give incorrect results. Artificial neural networks (ANN's) are recent developments in computer vision and image analysis. These networks are specialized computer software that generate a strategy to produce nonlinear mapping functions for complex problems. ANN's are commonly used as a tool for recognizing an object or predicting an event given an associated pattern. Only a limited number of applications of ANN for analyzing well-test data have been reported. These applications are mostly model-specific (developed for specific reservoir models) and, hence, are not general enough. This paper presents a new method based on ANN's that uses Ko-honen's self-organizing feature (SOF) mapping technique to identify well-test interpretation models. By grouping well-test data into distinct categories, the SOF algorithm produces a general mapping function. This method can help analyze well-test data from a large variety of reservoirs (including reservoirs with faults, fractures, boundaries, etc.) more efficiently and inexpensively than was feasible previously.

<!--[if gte mso 10]> <mce:style><! /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Norm&aacute;l t&aacute;bl&aacute;zat"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-qformat:yes; mso-style-parent:""; mso-padding-alt:0cm 5.4pt 0cm 5.4pt; mso-para-margin:0cm; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-fareast-font-family:"Times New Roman"; mso-fareast-theme-font:minor-fareast; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi;} > <! [endif] ><p class="MsoNormal" style="text-align: justify; line-height: 200%;" _mce_style="text-align: justify; line-height: 200%;"><span style="font-size: 10pt; line-height: 200%; font-family: ";Arial";,";sans-serif";;" lang="EN-US">Traditional techniques of identification of a depositional body from core data are costly and sometimes difficult to extrapolate to uncored wells. However, application of Kohonen&rsquo;s Self Organized Map (SOM) approach may be regarded to be a potential method for pattern recognition problems. A combination of Indicator Kriging and SOM for log-porosity and sand content data coming from quantitative well-log interpretations is used for identifying the spatial pattern of some delta-plain sub-environments. The basic high-dimensional property fields are defined by 3D shapes of well known depositional facieses. Many parameters as log-porosity and sand content data can be used to determine geo-property as a lithological pattern using SOM. This step of method can discover spatial patterns as clusters in unstructured data set because SOM is based on clustering algorithm. However, this approach not necessarily makes sure, that the resulting disjunctive clusters can show any meaningful depositional geometry. So at last the final geometry is given using Indicator Kriging method, which uses threshold values derived from property values of clusters.</span><-->Traditional techniques to identify a depositional body from core data are costly and sometimes difficult to extrapolate to uncored wells. The application of Kohonen&rsquo;s Self Organized Map (SOM) approach may be useful for the interpretation of a depositional rock body through well-log data. SOM is based on a clustering algorithm and this method can be used to discover spatial patterns occurring as clusters in unstructured data sets. An example of the application of SOM is presented whereby clusters through SOM can indicate the contours of well-known depositional patterns such as sub-environments.