A meta-cognitive sequential learning algorithm for neuro-fuzzy inference system

This research work focuses on the development of meta-cognitive sequential learning algorithms in Radial Basis Function (RBF) network classiers, and their application to the early diagnosis of neurodegenerative diseases. The important issues in existing sequential learning algorithms are proper selection of training samples, nding minimal network structure and selection of an appropriate learning strategy. In addition, the random sequence of sample arrival inuences the performance signicantly. It has been reported in human learning that best learning strategies employ meta-cognition (metacognition means cognition about cognition) to address fundamental problems of what-tolearn, when-to-learn and how-to-learn. This thesis develops such meta-cognitive sequential learning algorithms in RBF network for classication problems. We call a RBF network employing meta-cognitive algorithm as `meta-cognitive RBF network' (McRBFN). McRBFN is developed based on Nelson and Narens model of meta-cognition for human learning. Accordingly, McRBFN has two components, namely cognitive and meta-cognitive components. A RBF network with evolving structure is the cognitive component and a self-regulatory learning mechanism is its meta-cognitive component. List of Figures 1.1 Nelson and Narens model of meta-cognition . . . . . . . . . . . . . . . .

Neuro-fuzzy systems are learning machines that employ algorithms derived from articial neural networks to nd the parameters of a fuzzy inference system. These hybridintelligent systems can learn fuzzy rules from the data, while preserving their semantic properties. The learning algorithms employed in these systems are inspired from the physiology and functioning of human brain. As the study on neuroscience and cognitive psychology is an ongoing process, it is essential that these machine learning algorithms be extended with developments from the ndings in human learning psychology. Studies in human learning psychology have shown that self-regulated learning in a meta-cognitive framework is the best learning strategy. Meta-cognition also provides a learner with self-monitoring, which is a step-by-step process of evaluation during the learning process. In this thesis, we develop a Meta-Cognitive sequential learning algorithm for Neuro-Fuzzy Inference System called McFIS which is based on a well-known model of meta-cognition proposed by Nelson and Narens. Similar to Nelson and Narens model of meta-cognition, McFIS consists of a cognitive and a meta-cognitive component. A Takagi-Sugeno-Kang type neuro-fuzzy inference system forms the cognitive component and a self-regulatory learning mechanism is its meta-cognitive component. The metacognitive component monitors the knowledge in the cognitive component and controls the learning by eciently deciding on which sample to learn, when to learn it and how to learn it, eciently. Such a learning strategy helps the network generalize the functional relationship between input and output. The performance evaluation of McFIS on a set of benchmark function approximation and time-series prediction problems indicate signicant improvement over other state-of-the-art techniques. In addition, quantitative study on a set classication problems indicate motivating results. ix

In this paper, we propose a Projection Based Learning (PBL) algorithm for a Meta-Cognitive Neuro-Fuzzy Inference (McFIS) together referred to as PBL-McFIS. McFIS consists of a cognitive component, which is a zero-order Takagi-Sugeno-Kang adaptive neuro-fuzzy inference system, and a meta-cognitive component, which is a self-regulatory learning mechanism for the neuro-fuzzy inference system. The learning in the cognitive component begins with zero rules, and as new samples are presented to the network, the meta-cognitive component monitors the hinge-loss error and spherical potential of the current sample to efficiently decide on what-to-learn, when-to-learn and how-to-learn. In this work we employ PBL-McFIS to solve classification problems and hence the monitory signals employ class-specific self-adaptive thresholds to decide on efficient learning strategies. These thresholds are self-adapted such that the trained network is compact and avoids over-fitting. During addition of new rules or updating of existing rules, the optimal output weights corresponding to the minimum hinge-loss error is computed using PBL algorithm. The learning algorithm considers class-specific as well as class overlap factors during training. The performance of PBL-McFIS is evaluated on a set of benchmark classification problems. The statistical performance analysis with other state-of-the-art neuro-fuzzy inference systems and SVM indicate improved classification ability of the proposed algorithm.

Meta-cognitive Neural Network for classification problems in a sequential learning framework

This paper addresses the key problems of nonstationarity and influence of artifacts in electroencephalogram (EEG)-based brain-computer-interface (BCI) systems. The nonstationary nature of EEG data arises due to the physiological/instrumental differences in intra/intersession of the data generation process. This paper proposes a robust common spatial pattern feature extraction algorithm (RoCSP) to overcome the effects of artifacts and a self-regulated interval type-2 neuro-fuzzy inference system (SRIT2NFIS) to handle this inherent nonstationarity. Combined together, this approach is referred to as RoCSP-SRIT2NFIS. The RoCSP algorithm provides better features than the CSP algorithm by excluding those trials that are affected by the artifacts. SRIT2NFIS uses the features generated by the RoCSP algorithm as input and handles the nonstationarity as an uncertainty using the interval type-2 fuzzy sets in the antecedent of fuzzy rules. A self-regulatory learning mechanism is used to evolve the structure automatically and learn the parameters of the network. A regularized projection-based learning algorithm and a modified rule addition criterion are also proposed to improve the generalization performance of SRIT2NFIS. Using benchmark datasets, performance evaluation has been carried out, and the results indicate that, compared with other existing algorithms, RoCSP-SRIT2NFIS produces higher classification accuracy of 3-5% in simple tasks like left-right classification and 6-8% in complex tasks like foot-tongue classification. In addition, a statistical analysis of the performance results indicates that RoCSP-SRIT2NFIS performs better and is more suitable for an efficient BCI.

Domain-Specific Context of Studentś Self-Regulated Learning in the Preparation of Helping Professions

The present study discusses the mechanism and formalism of conversational planning. It is argued that the planner engages in a variety of metacognition as s/he perceives, interprets, evaluates, and produces cognitive actions that require careful attention and analysis. Following Flavell's (1981) model of cognitive monitoring, a metacognitive model of conversational planning is proposed. The model consists of several cognitive and metacognitive components that are grouped into three categories: Causal Knowledge, Working Knowledge, and Conscious Experiences. Next, to elaborate the metacognitive components of the model, several examples of metacognitive variables in problem-solving situations are proposed. Then, focusing on these variables, the proposed model of conversational planning is illustrated using natural data taken from a clinical supervision session. In the last section, the relevance of the present study to pragmatics is briefly discussed with respect to mental operations involved in interpretation.

In this paper, we develop an online sequential learning algorithm for single hidden layer feedforward networks (SLFNs) with additive or radial basis function (RBF) hidden nodes in a unified framework. The algorithm is referred to as online sequential extreme learning machine (OS-ELM) and can learn data one-by-one or chunk-by-chunk (a block of data) with fixed or varying chunk size. The activation functions for additive nodes in OS-ELM can be any bounded nonconstant piecewise continuous functions and the activation functions for RBF nodes can be any integrable piecewise continuous functions. In OS-ELM, the parameters of hidden nodes (the input weights and biases of additive nodes or the centers and impact factors of RBF nodes) are randomly selected and the output weights are analytically determined based on the sequentially arriving data. The algorithm uses the ideas of ELM of Huang et al. developed for batch learning which has been shown to be extremely fast with generalization performance better than other batch training methods. Apart from selecting the number of hidden nodes, no other control parameters have to be manually chosen. Detailed performance comparison of OS-ELM is done with other popular sequential learning algorithms on benchmark problems drawn from the regression, classification and time series prediction areas. The results show that the OS-ELM is faster than the other sequential algorithms and produces better generalization performance.

In this article, a metacognitive sedenion-valued neural network (Mc-SVNN) and its learning algorithm are proposed. Its application to diverse time-series prediction problems is presented. The Mc-SVNN contains two components: a sedenion-valued neural network that represents the cognitive component, and a metacognitive component, which serves to self-regulate the learning algorithm. At each epoch, the metacognitive component decides what, how, and when learning occurs. The algorithm deletes unnecessary samples and stores only those that are used. This decision is determined by the sedenion magnitude and the 15 sedenion phases. The Mc-SVNN is applied to four real-world forecasting problems: USD-to-euro currency exchange rate forecasting, the sunspot number time series, power demand forecasting, and daily temperature prediction in Abu Dhabi. Compared to existing methods, the Mc-SVNN demonstrates superior performance in time-series forecasting while using a smaller number of parameters.

Magix—an ICAE system for Problem-Based Learning

A neuro-fuzzy classifier based on the meta-cognitive principle of human self-regulated learning (Mc-FIS) is proposed in this paper. The network decides what-to-learn, when-to-learn and how-to-learn based on the current information present in the classifier and the new information present in the sample. The classifier utilizes self-regulating error based criterion to decide which sample to learn and when to learn. A rule is pruned if its significance is below a particular threshold, based on class specific information. This results in a compact network and sample deletion helps overfitting. Class specific information is used in executing the above tasks. The algorithm is evaluated on balanced and unbalanced benchmark problems from UCI machine learning repository. The results clearly indicate the superiority of the developed algorithm.

Many sequential algorithms have been proposed for the mining of association rules. However, very little work has been done in mining association rules in distributed databases. A direct application of sequential algorithms to distributed databases is not effective, because it requires a large amount of communication overhead. In this study, an efficient algorithm called DMA (Distributed Mining of Association rules), is proposed. It generates a small number of candidate sets and requires only O(n) messages for support-count exchange for each candidate set, where n is the number of sites in a distributed database. The algorithm has been implemented on an experimental testbed, and its performance is studied. The results show that DMA has superior performance, when compared with the direct application of a popular sequential algorithm, in distributed databases.

Data partitioning algorithms aiming to minimize the execution time and the energy of computations in self-adaptable data-parallel applications on modern extreme-scale multicore platforms must address two critical challenges. First, they must take into account the new complexities inherent in these platforms such as severe resource contention and non-uniform memory access. Second, they must have low practical runtime and memory costs. The sequential data partitioning algorithms addressing the first challenge have a theoretical time complexity of O( $m * m * p * p$ ) where $m$ is the number of points in the discrete speed/energy function and $p$ is the number of available processors. They, however, exhibit high practical runtime cost and excessive memory footprint, therefore, rendering them impracticable for employment in self-adaptable applications executing on extreme-scale multicore platforms. We present, in this paper, the parallel data partitioning algorithms that address both the challenges. They take as input the functional models of performance and energy consumption against problem size and output workload distributions, which are globally optimal solutions. They have a low time complexity of O( $m * m * p$ ) thereby providing a linear speedup of O( $p$ ) and low memory complexity of O( $n$ ) where $n$ is the workload size expressed as a multiple of granularity. They employ dynamic programming approach, which also facilitates the easier integration of performance and energy models of communications. We experimentally study the practical cost of application of our algorithms in two data-parallel applications, matrix multiplication and fast Fourier transform, on a cluster in Grid’5000 platform. We demonstrate that their practical runtime and memory costs are low making them ideal for employment in self-adaptable applications. We also show that the parallel algorithms exhibit tremendous speedups over the sequential algorithms. Finally, using theoretical analysis for a forecast exascale platform, we demonstrate that the parallel algorithms have negligible execution times compared to the matrix multiplication application executing on the platform.

Publication in the conference proceedings of EUSIPCO, Florence, Italy, 2006

Bond rating is regarded as an important event for measuring financial risk of companies and for determining the investment returns of investors. As a result, it has been a popular research topic for researchers to predict companies' credit ratings by applying statistical and machine learning techniques. The statistical techniques, including multiple regression, multiple discriminant analysis (MDA), logistic models (LOGIT), and probit analysis, have been traditionally used in bond rating. However, one major drawback is that it should be based on strict assumptions. Such strict assumptions include linearity, normality, independence among predictor variables and pre-existing functional forms relating the criterion variablesand the predictor variables. Those strict assumptions of traditional statistics have limited their application to the real world. Machine learning techniques also used in bond rating prediction models include decision trees (DT), neural networks (NN), and Support Vector Machine (SVM). Especially, SVM is recognized as a new and promising classification and regression analysis method. SVM learns a separating hyperplane that can maximize the margin between two categories. SVM is simple enough to be analyzed mathematical, and leads to high performance in practical applications. SVM implements the structuralrisk minimization principle and searches to minimize an upper bound of the generalization error. In addition, the solution of SVM may be a global optimum and thus, overfitting is unlikely to occur with SVM. In addition, SVM does not require too many data sample for training since it builds prediction models by only using some representative sample near the boundaries called support vectors. A number of experimental researches have indicated that SVM has been successfully applied in a variety of pattern recognition fields. However, there are three major drawbacks that can be potential causes for degrading SVM's performance. First, SVM is originally proposed for solving binary-class classification problems. Methods for combining SVMs for multi-class classification such as One-Against-One, One-Against-All have been proposed, but they do not improve the performance in multi-class classification problem as much as SVM for binary-class classification. Second, approximation algorithms (e.g. decomposition methods, sequential minimal optimization algorithm) could be used for effective multi-class computation to reduce computation time, but it could deteriorate classification performance. Third, the difficulty in multi-class prediction problems is in data imbalance problem that can occur when the number of instances in one class greatly outnumbers the number of instances in the other class. Such data sets often cause a default classifier to be built due to skewed boundary and thus the reduction in the classification accuracy of such a classifier. SVM ensemble learning is one of machine learning methods to cope with the above drawbacks. Ensemble learning is a method for improving the performance of classification and prediction algorithms. AdaBoost is one of the widely used ensemble learning techniques. It constructs a composite classifier by sequentially training classifiers while increasing weight on the misclassified observations through iterations. The observations that are incorrectly predicted by previous classifiers are chosen more often than examples that are correctly predicted. Thus Boosting attempts to produce new classifiers that are better able to predict examples for which the current ensemble's performance is poor. In this way, it can reinforce the training of the misclassified observations of the minority class. This paper proposes a multiclass Geometric Mean-based Boosting (MGM-Boost) to resolve multiclass prediction problem. Since MGM-Boost introduces the notion of geometric mean into AdaBoost, it can perform learning process considering the geometric mean-based accuracy and errors of multiclass. This study applies MGM-Boost to the real-world bond rating case for Korean companies to examine the feasibility of MGM-Boost. 10-fold cross validations for threetimes with different random seeds are performed in order to ensure that the comparison among three different classifiers does not happen by chance. For each of 10-fold cross validation, the entire data set is first partitioned into tenequal-sized sets, and then each set is in turn used as the test set while the classifier trains on the other nine sets. That is, cross-validated folds have been tested independently of each algorithm. Through these steps, we have obtained the results for classifiers on each of the 30 experiments. In the comparison of arithmetic mean-based prediction accuracy between individual classifiers, MGM-Boost (52.95%) shows higher prediction accuracy than both AdaBoost (51.69%) and SVM (49.47%). MGM-Boost (28.12%) also shows the higher prediction accuracy than AdaBoost (24.65%) and SVM (15.42%)in terms of geometric mean-based prediction accuracy. T-test is used to examine whether the performance of each classifiers for 30 folds is significantly different. The results indicate that performance of MGM-Boost is significantly different from AdaBoost and SVM classifiers at 1% level. These results mean that MGM-Boost can provide robust and stable solutions to multi-classproblems such as bond rating.