Centroid neural network for unsupervised competitive learning

A LEARNING ALGORITHM WITH MULTIPLE CRITERIA FOR SELF-ORGANIZING FEATURE MAPS

BACKGROUNDTraditional methods of developing predictive models in inflammatory bowel diseases (IBD) rely on using statistical regression approaches to deriving clinical scores such as the Crohn's disease (CD) activity index. However, traditional approaches are unable to take advantage of more complex data structures such as repeated measurements. Deep learning methods have the potential ability to automatically find and learn complex, hidden relationships between predictive markers and outcomes, but their application to clinical prediction in CD and IBD has not been explored previously.AIMTo determine and compare the utility of deep learning with conventional algorithms in predicting response to anti-tumor necrosis factor (anti-TNF) therapy in CD.METHODSThis was a retrospective single-center cohort study of all CD patients who commenced anti-TNF therapy (either adalimumab or infliximab) from January 1, 2010 to December 31, 2015. Remission was defined as a C-reactive protein (CRP) < 5 mg/L at 12 mo after anti-TNF commencement. Three supervised learning algorithms were compared: (1) A conventional statistical learning algorithm using multivariable logistic regression on baseline data only; (2) A deep learning algorithm using a feed-forward artificial neural network on baseline data only; and (3) A deep learning algorithm using a recurrent neural network on repeated data. Predictive performance was assessed using area under the receiver operator characteristic curve (AUC) after 10× repeated 5-fold cross-validation.RESULTSA total of 146 patients were included (median age 36 years, 48% male). Concomitant therapy at anti-TNF commencement included thiopurines (68%), methotrexate (18%), corticosteroids (44%) and aminosalicylates (33%). After 12 mo, 64% had CRP < 5 mg/L. The conventional learning algorithm selected the following baseline variables for the predictive model: Complex disease behavior, albumin, monocytes, lymphocytes, mean corpuscular hemoglobin concentration and gamma-glutamyl transferase, and had a cross-validated AUC of 0.659, 95% confidence interval (CI): 0.562-0.756. A feed-forward artificial neural network using only baseline data demonstrated an AUC of 0.710 (95%CI: 0.622-0.799; P = 0.25 vs conventional). A recurrent neural network using repeated biomarker measurements demonstrated significantly higher AUC compared to the conventional algorithm (0.754, 95%CI: 0.674-0.834; P = 0.036).CONCLUSIONDeep learning methods are feasible and have the potential for stronger predictive performance compared to conventional model building methods when applied to predicting remission after anti-TNF therapy in CD.

A competitive exception learning algorithm (CELA) is proposed which derives exceptional patterns from a given set of data pairs. In the first step of the algorithm, the output space is fuzzily partitioned (using a competitive learning algorithm) and then characterized by a so-called unconditional cluster membership distribution. In the second step, the input space is fuzzily partitioned as well by again using competitive learning. After learning, the input clusters represent locations in the input space which correspond to the most exceptional patterns in the output space. These patterns are characterized by so-termed conditional cluster membership distributions. In the third step, a fuzzy rule base is set up describing the exceptions in linguistic terms. In addition, the complete fuzzy input-output map can be estimated using a mathematical function. Besides describing the various steps of the new algorithm, a theoretical framework is introduced using an information-theoretical point of view: all output data points are supposed to be generated by a non-stationary stochastic, fuzzy information source. A summary concerning the results of certain simulations completes the description of CELA. The article is concluded with a discussion and outlook.

In this paper, a novel unsupervised competitive learning algorithm, called the centroid neural network adaptive resonance theory (CNN-ART) algorithm, is to be proposed to relieve the dependence on the initial codewords of the codebook in contrast to the conventional algorithms with vector quantization in lossy image compression. The design of the CNN-ART algorithm is mainly based on the adaptive resonance theory (ART) structure, and then a gradient-descent based learning rule is derived so that the CNN-ART algorithm does not require a predetermined schedule for learning rate. The appropriate initial weights obtained from the CNN-ART algorithm can be applied as an initial codebook of the Linde-Buzo-Gray (LBG) algorithm such that the compression performance can be greatly improved. In this paper, the extensive simulations demonstrate that the CNN-ART algorithm does outperform other algorithms Re LBG, SOFM and DCL.

Autonomous underwater vehicles (AUVs) have great advantages for activities in deep sea, and expected as the attractive tool. However, AUVs have various problems which should be solved. In this paper, the Self-Organizing Map (SOM) is applied as the clustering method for the navigation system. The SOM is known as one of the effective methods to extract the principle feature from many parameters and decrease the dimension of parameters. Through the competitive learning algorithms, the obtained is tuned to express specific features of the input signals. We have been investigating the possibility of navigation system based on SOM through simulations are experiments with an AUV called Twin-Burger. The learning algorithm of usual SOM is unsupervised learning. However, supervised learning algorithms should be introduced because the relationship between distances information and desirable behavior of the robot, that is, the relationship from inputs to outputs should be acquired and learned. In this paper, a supervised learning algorithm is introduced into SOM and a method to adapt the local to its environment by learning and evaluating the trajectory of robot is proposed. In the proposed method, the map is made static and digital value as teaching data. In order to include more information of environment in the initial map, the trajectories of robot are evaluated, and the evaluation is utilized in the learning process. This method enables the to have both the effect of dynamics of robot and environmental information. The efficiency of the method is investigated through the simulations and experiments.

An unsupervised competitive learning rule, called the vectorial boundary adaptation rule (VBAR), is introduced for topographic map formation. Since VBAR is aimed at producing an equiprobable quantization of the input space, it yields a nonparametric model of the input probability density function. Furthermore, since equiprobable quantization is equivalent to unconditional entropy maximization, we argue that this is a plausible strategy for maximizing mutual information (Shannon information rate) in the case of "online" learning. We use mutual information as a tool for comparing the performance of our rule with Kohonen's self-organizing (feature) map algorithm.

A classification scheme for content-based audio signal retrieval is proposed in this paper. The proposed scheme uses the Centroid Neural Networks (CNN) with a Divergence Measure called Divergence-based Centroid Neural Network (DCNN) to perform clustering of Gaussian Probability Density Function (GPDF) data. In comparison with other conventional algorithms, the DCNN designed for probability data has the robustness advantages of utilizing a audio data representation method in which each audio data is represented by a Gaussian distribution feature vector. Experiments and results for several audio data sets have shown that the DCNN-based classification algorithm has accuracy improvements over models employing the conventional k-means and Self Organizing Map (SOM) algorithms.

In this paper, a novel neural-fuzzy controller based on fuzzy differential competitive learning is proposed. Since one of the most important parts is the generation of the fuzzy rules in the design of the fuzzy control system, a fast learning algorithm, fuzzy differential competitive learning (FDCL), for the generation of the rules is applied in the fuzzy control system. The FDCL algorithm adopts a principle of learn according to how well it wins. Unlike the previous competitive learning algorithm such as crisp competitive learning algorithms where only one neuron will win and learn at each competition step every neuron in the neural network based on FDCL algorithm will along with its different distance to the input pattern and learns the pattern accordingly. Compared with the ordinary competitive learning algorithm the proposed FDCL algorithm has the various distinguishing features. The FDCL algorithm is implanted in the neural network based fuzzy system and the network adopted is fuzzy associative memory system (FAMS) which simulates the knowledge representation and inference process by using fuzzy notation and by association in neural networks. In FAMS the fuzzy rules will be generated by clustering the input-output training data through the FDCL paradigm. By using the FDCL algorithm the neural network can highly refine knowledge and represent the expert experience.

Among the many research problems that appeal to the scientific community toward smart, IoT-enabled developments, human activity recognition has always been relevant and interesting. This human activity recognition of active research offers several enticing challenges, such as single person activity recognition, group activity recognition, monitoring and tracking toward indoor healthcare, and room occupancy estimation. Out of these, room occupancy classification from passive data is an important one. While most works focus on the use of conventional learning or deep learning algorithms toward developing an automated solution, the recent progress in fuzzy-set and rough-set theory can be advantageous in solving data-driven problems. The proposed work leverages this mathematical complexity of the convergence of fuzzy-rough set theories along a conventional machine learning algorithm to provide a smart solution to the problem of room occupancy estimation. The experiments reveal that the conventional learning algorithm alone is outperformed by the chosen fuzzy-rough set theory-based classification approach, with a high accuracy of 99.96%.KeywordsRoom occupancy estimationHuman activity recognitionRough set theoryFuzzy set theorySupervised learningFuzzy-rough set

This paper presents a comparison of competitive learning algorithms for Self Organizing Map (SOM). The competitive learning algorithms showed to self organizing map algorithm are winner-takes-all, Frequency Sensitive Competitive Learning and Rival Penalized Competitive Learning. The result shows the performance in classification of partial discharge on power cables using SOM.

Motivated by earlier work on adaptive event forecasting, this paper proposes an iterative event signature extraction method for wireless sensor networks, and a probabilistic approach to model non i.i.d. (independent and identically distributed) aperiodic traffic. The model is validated on well described (collection included) real world measurements of various types, and it is then used to demonstrate the effectiveness of the proposed signature extraction method to support reliable event forecasting. Our scheme can continuously keep the event signature database low on artifacts, dynamically estimate the number of sequences and extract the signatures from noisy, overlapped events. The proposed solution is inspired by unsupervised competitive Hebbian learning used in self-organizing Kohonen maps. We evaluate the proposed solution analytically, but also empirically, by means of simulations.

Some applications of neural network algorithms for the design of vector quantizer codebooks are presented. Vector quantization has been applied to the problem of displaying natural images with a reduced set of colors (color palette) and to the interframe coding of image sequences. Self-organizing feature maps and competitive learning (CL) algorithms have been used for the codebook design and their results are compared with those obtained using a classical LBG (Linde, Buzo, Gray) algorithm. The best results are obtained using a CL algorithm with a new initialization strategy. With this technique the codebook design is very fast, and some experiments have been carried out in order to introduce adaptive vector quantization in the interframe coder. Frame by frame the codebooks used to quantize the most important information are updated and then the more active parts of the motion-compensated luminance differences are quantized with these new codebooks. A considerable rise of the coder performance was obtained. >

Fault detection in linear systems is a fairly matured area where the well-known principal component analysis (PCA) and its variants are widely used. However, a large class of non-linear systems exist, especially chemical processes, on which such techniques cannot be applied. The present work aims at demonstrating the application of self-organising maps (SOM) for fault detection in non-linear processes. SOM belongs to the class of unsupervised and competitive learning algorithms and it is highly capable of handling nonlinear relationships. Application of SOM to fault detection involves generation of a reference template for the process under fault-free conditions. Online fault detection is performed by generating a new template using a windowing of the data, which is compared with the reference template using a novel metric based on the node weights obtained from SOM to detect possible faults in the process. Simulation studies on two non-linear systems, namely, (1) continuously stirred tank reactor (CSTR) and (2) bioreactor process demonstrate the practicality and utility of the proposed method.

Training of multilayered feed-forward neural networks (MLFFNNs) is considered in this work.A procedure to derive high performance learning algorithm for updating the network weights is proposed.The proposed algorithm is based on heuristic technique that is developed from an analysis of the performance of the basicbackpropagation training algorithm.A unified formulation of the conventional learning algorithms including the basic-backpropagation algorithm, the momentum algorithm, and the exponential-smoothing algorithm alongside with the proposed learning algorithm is introduced.Recursive relations for updating the weights of the network are derived which greatly simplifies the application of these rules.Simulation results are presented and comparative studies are carried out to demonstrate the effectiveness of the new learning algorithm.The new algorithm can converge more than hundred times faster than the conventional algorithms.

For the classical k-means clustering algorithm, the problem of selecting an appropriate k is a hard problem and affects the performance of k-means strongly. When used for clustering analysis, the conventional competitive learning (CL) algorithms also have a similar crucial problem-the selection of an appropriate number of neural units. The performance of frequency sensitive competitive learning (FSCL)-one version of the improved CL algorithms, also significantly deteriorates when the number of units is inappropriately selected. The paper proposes a new algorithm called rival penalized competitive learning (RPCL), which has the ability of automatically allocating an appropriate number of units for an input data set. The experimental results have shown that RPCL outperforms FSCL significantly when they are used for unsupervised classification, and supervised classification through the radial basis function net. >