Supervised Learning Classifier System for Grid Data Mining

Data mining, and specifically supervised data classification, is a key application area for Learning Classifier Systems (LCS). Scaling to larger classification problems, especially to higher dimensional problems, is a key challenge. Ensemble based approaches can be applied to LCS to address scalability issues. To this end a rough set based ensemble of LCS is proposed, which relies on a pre-processed feature partitioning step to train multiple LCS on feature subspaces. Each base classifier in the ensemble is a Michigan style supervised LCS. The traditional genetic algorithm based rule evolution is replaced by a differential evolution based rule discovery, to improve generalisation capabilities of LCS. A voting mechanism is then used to generate output for test instances. This paper describes the proposed ensemble algorithm in detail, and compares its performance with different versions of base LCS on a number of benchmark classification tasks. Analysis of computational time and model accuracy show the relative merits of the ensemble algorithm and base classifiers on the tested data sets. The rough set based ensemble learning approach and differential evolution based rule searching out-perform the base LCS on classification accuracy over the data sets considered. Results also show that small ensemble size is sufficient to obtain good performance.

<p>This thesis introduces a Three-Cornered Coevolution System that is capable of addressing classification tasks through coevolution (coadaptive evolution) where three different agents (i.e. a generation agent and two classification agents) learn and adapt to the changes of the problems without human involvement. In existing pattern classification systems, humans usually play a major role in creating and controlling the problem domain. In particular, humans set up and tune the problem’s difficulty. A motivation of the work for this thesis is to design and develop an automatic pattern generation and classification system that can generate various sets of exemplars to be learned from and perform the classification tasks autonomously. The system should be able to automatically adjust the problem’s difficulty based on the learners’ ability to learn (e.g. determining features in the problem that affect the learners’ performance in order to generate various problems for classification at different levels of difficulty). Further, the system should be capable of addressing the classification tasks through coevolution (coadaptive evolution), where the participating agents learn and adapt to the changes of the problems without human participation. Ultimately, Learning Classifier System (LCS) is chosen to be implemented in the participating agents. LCS has several potential characteristics, such as interpretability, generalisation capability and variations in representation, that are suitable for the system. The work can be broken down into three main phases. Phase 1 is to develop an automated evolvable problem generator to autonomously generate various problems for classification, Phase 2 is to develop the Two-Cornered Coevolution System for classification, and Phase 3 is to develop the Three-Cornered Coevolution System for classification. Phase 1 is necessary in order to create a set of problem domains for classification (i.e. image-based data or artificial data) that can be generated automatically, where the difficulty levels of the problem can be adjusted and tuned. Phase 2 is needed to investigate the generation agent’s ability to autonomously tune and adjust the problem’s difficulty based on the classification agent’s performance. Phase 2 is a standard coevolution system, where two different agents evolve to adapt to the changes of the problem. The classification agent evolves to learn various classification problems, while the generation agent evolves to tune and adjust the problem’s difficulty based on the learner’s ability to learn. Phase 3 is the final research goal. This phase develops a new coevolution system where three different agents evolve to adapt to the changes of the problem. Both of the classification agents evolve to learn various classification problems, while the generation agent evolves to tune and adjust the problem’s difficulty based on the classification agents’ ability to learn. The classification agents use different styles of learning techniques (i.e. supervised or reinforcement learning techniques) to learn the problems. Based on the classification agents’ ability (i.e. the difference in performance between the classification agents) the generation agent adjusts and creates various problems for classification at different levels of difficulty (i.e. various ‘hard’ problems). The Three-Cornered Coevolution System offers a great potential for autonomous learning and provides useful insight into coevolution learning over the standard studies of pattern recognition. The system is capable of autonomously generating various problems, learning and providing insight into each learning system’s ability by determining the problem domains where they perform relatively well. This is in contrast to humans having to determine the problem domains.</p>

Learning classifier systems (LCSs) are rule-based inductive learning systems that have been widely used in the field of supervised and reinforcement learning over the last few years. This paper employs sUpervised Classifier System (UCS), a supervised learning classifier system, that was introduced in 2003 for classification tasks in data mining. We present an adaptive framework of UCS on top of a self-organized map (SOM) neural network. The overall classification problem is decomposed adaptively and in real time by the SOM into subproblems, each of which is handled by a separate UCS. The framework is also tested with replacing UCS by a feedforward artificial neural network (ANN). Experiments on several synthetic and real data sets, including a very large real data set, show that the accuracy of classifications in the proposed distributed environment is as good or better than in the nondistributed environment, and execution is faster. In general, each UCS attached to a cell in the SOM has a much smaller population size than a single UCS working on the overall problem; since each data instance is exposed to a smaller population size than in the single population approach, the throughput of the overall system increases. The experiments show that the proposed framework can decompose a problem adaptively into subproblems, maintaining or improving accuracy and increasing speed.

Just-in-time software defect prediction (JIT-SDP) is an active topic in software defect prediction, which aims to identify defect-inducing changes. Recently, some studies have found that the variability of defect data sets can affect the performance of defect predictors. By using local models, it can help improve the performance of prediction models. However, previous studies have focused on module-level defect prediction. Whether local models are still valid in the context of JIT-SDP is an important issue. To this end, we compare the performance of local and global models through a large-scale empirical study based on six open-source projects with 227417 changes. The experiment considers three evaluation scenarios of cross-validation, cross-project-validation, and timewise-cross-validation. To build local models, the experiment uses the k-medoids to divide the training set into several homogeneous regions. In addition, logistic regression and effort-aware linear regression (EALR) are used to build classification models and effort-aware prediction models, respectively. The empirical results show that local models perform worse than global models in the classification performance. However, local models have significantly better effort-aware prediction performance than global models in the cross-validation and cross-project-validation scenarios. Particularly, when the number of clusters k is set to 2, local models can obtain optimal effort-aware prediction performance. Therefore, local models are promising for effort-aware JIT-SDP.

In response to the deficiencies in visible-near infrared spectral identification of coal gangue under different working conditions, this paper employs multiple algorithms to construct local and global recognition models and conducts analysis. Simulating five different working conditions in the laboratory, including different detection angles, illumination angles, detection distances, coal powder content, and moisture content. Each working condition is further divided into four levels. Collecting spectral data of coal and gangue samples. Constructing global and local datasets, which are respectively derived from all levels and single levels under certain working conditions. Training local and global models to identify samples based on machine learning algorithms such as Lightgbm and 1DCNN. The results of the experiment indicate that there are variations in the sample spectra and recognition accuracy based on different working conditions. Furthermore, the local model exhibits lower recognition accuracy and inadequate robustness for other level samples under the same working condition. The global model is more robust and practical, as it achieves higher or equal recognition accuracy for local data compared to the local model. Among the global models, Lightgbm has the best overall performance, with an average F1-score of 99.34% and 97.9% for local and global datasets, respectively. It also has fast training time of 0.75s and recognition time of 1 ms. The Lightgbm global model can accurately identify coal and gangue in various working conditions, making it a reliable solution.

Learning classifier systems (LCS) are algorithms that incorporate genetic algorithms with reinforcement learning to produce adaptive systems described by if-then rules. As a new interdisciplinary branch of biology, synthetic biology pursues the design and construction of complex artificial biological systems from the bottom-up. A trend is growing in designing artificial metabolic pathways that show previously undescribed reactions produced by the assembly of enzymes from different sources in a single host. However, few researchers have succeeded thus far because of the difficulty in analyzing gene expression. To tackle this problem, data mining and knowledge discovery are essential. In this context, nature-inspired LCS are well suited to extracting knowledge from complex systems and thus can be exploited to investigate and utilize natural biological phenomena. This chapter focuses on applying LCS to gene expression analysis in synthetic biology. Specifically, it describes the optimization of artificial operon structure for the biosynthesis of metabolic pathway products in Escherichia coli. This optimization is achieved by manipulating the order of multiple genes within the artificial operons.

Learning Classifier Systems (LCSs) are a class of expert systems that use a knowledge base of decision rules and a genetic algorithm (GA) [9] as a discovery mechanism. The set of decision rules allows the LCS to represent and learn control strategies, while the robust search ability of the GA allows it to search for new rules based on the performance of existing rules. LCS were first designed to solve machine learning problems, especially classification problems. Classification problems are problems where instances of a data set belong to a set of classes, and the system needs to infer, based on past experience, the correct class (or classes) of new, previously unseen, instances. However, the features of LCSs are also very useful for solving reinforcement learning problems, a class of problems where the system should learn to operate in the environment based only on performance feedback. This paper considers LCSs as an approach to classification problems, more specifically a more complex kind of classification called multi-label classification. This paper analyses the default hierarchy formation theory presented by [14] as a way of favoring the hierarchical arrangement of rules, and also the organizational learning theory [17] for adjusting the degree of individual and collective behaviors. We suggest a new method, combining both organizational learning and default hierarchy formation, for solving multi-label problems. The preliminary results with a simple multi-label problem show the potential of this method. Final discussion presents the conclusions and directions for further research.

A learning strategy in Learning Classifier Systems (LCSs) defines how classifiers cover a state-action space in a problem. Previous analyses in classification problems have empirically claimed an adequate learning strategy can be decided depending on the types of noise in the problem. This issue is still arguable from two aspects. First, there lacks comparison of learning strategies in reinforcement learning problems with different types of noise. Second, when we can claim so, a further issue is how should classifiers cover the state-action space in order to improve the stability of LCS performance on as many types of noise as possible? This paper first attempts to empirically conclude these issues on a version of LCSs (i.e., the XCS classifier system). That is, we present a new concept of learning strategy for LCSs, and complement that claim by comparing it with the existing learning strategies on a reinforcement learning problem. Our learning strategy covers all state-action pairs but assigns more classifiers to the highest-return action at each state than other actions. Our results support that claim that existing learning strategies have dependencies on the types of noise in reinforcement learning problems. However, our learning strategy improves the stability of XCS performance compared with the existing strategies on all types of noise employed in this paper.

Rough set theory has proved efficient for many applications, including finding hidden patterns in data, data reduction, evaluating significance of data, and generating sets of decision rules from data. Recently, it has shown to be effective approach for constructing ensemble learning systems as well. Learning classifier systems are genetics-based machine learning techniques that have recently shown a high degree of competence on a variety of data mining problems. Attempts to improve its generalization capabilities in the literature using ensemble learning lack a systematic and robust techniques for partitioning the problem at hand. It is well known that ensemble performance depends on the problem decomposition technique being used. Rough set based learning classifier system ensemble is proposed in this paper. In this approach, rough set attribute reduction is used to generate a set of reducts, and then a diverse subset of these reducts is selected to train an ensemble of base classifiers. The experiments show that classification accuracy of reduct-based ensemble systems outperforms a single learning classifier system model. It has also shown better performance than either an ensemble of classifiers with all attributes being used or a single classifier trained by a single reduct. It has also shown competitive performance to the random subspace ensemble strategy on the set of real data sets used in the experiments.

Evaluating a thermal image sharpening model over a mixed agricultural landscape in India

CALENDAR

Data heterogeneity across clients in federated learning (FL) settings is a widely acknowledged challenge. In response, personalized federated learning (PFL) emerged as a framework to curate local models for clients' tasks. In PFL, a common strategy is to develop local and global models jointly - the global model (for generalization) informs the local models, and the local models (for personalization) are aggregated to update the global model. A key observation is that if we can improve the generalization ability of local models, then we can improve the generalization of global models, which in turn builds better personalized models. In this work, we consider class imbalance, an overlooked type of data heterogeneity, in the classification setting. We propose FedNH, a novel method that improves the local models' performance for both personalization and generalization by combining the uniformity and semantics of class prototypes. FedNH initially distributes class prototypes uniformly in the latent space and smoothly infuses the class semantics into class prototypes. We show that imposing uniformity helps to combat prototype collapse while infusing class semantics improves local models. Extensive experiments were conducted on popular classification datasets under the cross-device setting. Our results demonstrate the effectiveness and stability of our method over recent works.

Additive manufacturing (AM) enables the production of customized and complex metal parts, providing greater flexibility than traditional mass production. However, maintaining consistent product quality for AM processes remains challenging due to process variability and heat emission fluctuations, which can cause defects such as overheating. Overheating occurs when non-uniform temperature distributes across the printed parts, leading to issues such as structural weaknesses. Predicting heat emissions accurately is essential to preventing such issues, but AM facilities, particularly small- and medium-sized enterprises (SMEs), generate limited and heterogeneous datasets due to unique configurations and privacy concerns, making traditional centralized predictive model training paradigms impractical. Federated learning (FL) offers a privacy-preserving solution, it alleviates data privacy and data scarcity concerns by enabling collaborative learning across decentralized datasets while keeping data local. However, its performance suffers under heterogeneous client data and varied computational resources. This creates a need to address data heterogeneity while maintaining both global model performance and individual client personalization. We propose FedScope-KD, a novel heterogeneity-aware FL framework designed to balance the scope of global and personalized learning. Client heterogeneity level is quantified prior to the FL training, to improve the model weight aggregation during training. This is achieved through component decomposition and knowledge distillation. The framework decomposes client data into process-invariant representations, which capture the fundamental nature of the laser powder bed fusion (LPBF) process, and process-variant representations, which reflect local, client-specific configuration settings. This decomposition ensures that global models benefit from collaborative learning on the homogeneous data representations, while personalized predictions remain tailored to client-specific heterogeneous data representations. A knowledge distillation (KD) mechanism further enhances the framework by efficiently transferring process-invariant knowledge from a global teacher model to local student models. We compare the heterogeneity index generation and calculate the Dynamic Time Warping (DTW) distance for both inter-client variant and invariant components. The variant components exhibit greater distances between clients, reflecting their inherent heterogeneity, while the invariant components are more closely aligned across clients. This pattern demonstrates the effectiveness of the decomposition process in separating heterogeneous and homogeneous information. The heterogeneity index is further used for weight aggregation during FL training. The framework’s performance is validated through a comparison with baseline FL methods such as FedAvg, as well as centralized and individual learning paradigms. Performance metrics include personalized performance and global accuracy with MSE. Through experiments on heat emission datasets in the LPBF process, we show that FedScope-KD successfully addresses the challenges of data heterogeneity and resource constraints, providing a robust solution of predictive modeling for SMEs.

Taking another step: A simple approach to high-dimensional Bayesian optimization

Estimating foliar phosphorus of rubber trees using locally modelling approach with hyperspectral reflectance