Transductive Transfer Learning in Genetic Programming for Document Classification

Transductive transfer learning based Genetic Programming for balanced and unbalanced document classification using different types of features

Document classification is a challenging task to the data being high-dimensional and sparse. Many transfer learning methods have been investigated for improving the classification performance by effectively transferring knowledge from a source domain to a target domain, which is similar to but different from the source domain. However, most of the existing methods cannot handle the case that the training data of the target domain does not have labels. In this study, we propose a transductive transfer learning system, utilizing solutions evolved by genetic programming (GP) on a source domain to automatically pseudolabel the training data in the target domain in order to train classifiers. Different from many other transfer learning techniques, the proposed system pseudolabels target-domain training data to retrains classifiers using all target-domain features. The proposed method is examined on nine transfer learning tasks, and the results show that the proposed transductive GP system has better prediction accuracy on the test data in the target domain than existing transfer learning approaches including subspace alignment-domain adaptation methods, feature-level-domain adaptation methods, and one latest pseudolabeling strategy-based method.

In this paper, we propose a transductive transfer learning framework, referred to as Transfer Maximum Margin Criterion (T-MMC). This framework is suitable to transfer the knowledge acquired in one domain, the source domain, to another domain, the target domain, where no labeled examples are available in the target domain. We introduce an e ective feature weighting approach, which proceeds to reduce the domain di erence between the source and target domains. Moreover, we exploit maximum margin criterion to well discriminate various classes in the reduced domains. We simultaneously transfer knowledge from the source domain to target domain and also discriminate various classes in the reduced domains. Comprehensive experiments on the synthetic and real datasets demonstrate that T-MMC outperforms existing transfer learning methods.

Gaussian-based edge detectors have been developed for many years, but there are still problems with how to set scales for Gaussian filters and how to combine Gaussian filters. In order to address both problems, a Genetic Programming (GP) system is proposed to automatically choose scales for Gaussian filters and automatically combine Gaussian filters. In this study, the GP system is utilised to construct rotation invariant Gaussian-based edge detectors based on a benchmark image dataset. The experimental results show that the GP evolved Gaussian-based edge detectors are better than the Gaussian gradient and rotation invariant surround suppression to extract edge features.

The 0/1 knapsack problem is one of the most exhaustively studied NP-hard combinatorial optimization problems. Many different approaches have been taken to obtain an approximate solution to the problem in polynomial time. Here we consider the biobjective 0/1 knapsack problem. The contribution of this paper is to show that a genetic programming system can evolve a set of heuristics that can give solutions on the Pareto front for multiobjective combinatorial problems. The genetic programming (GP) system outlined here evolves a heuristic which decides whether or not to add an item to the knapsack in such a way that the final solution is one of the Pareto optimal solutions. Moreover, the Pareto front obtained from the GP system is comparable to the front obtained from other human-designed heuristics. We discuss the issue of the diversity of the obtained Pareto front and the application of strongly-typed GP as a means of obtaining better diversity.

Algorithms are needed in every aspect of parallel computing. Genetic Programming is an evolutionary technique for automating the design of algorithms through iterative steps of mutation and crossover operations on an initial population of randomly generated computer programs. This paper describes a novel parallel genetic programming (GP) system inspired by the symbiogenesis model of evolution, wherein new organisms are generated through the absorption of different life-forms in addition to the usual mutation and crossover operations. Different organisms are expressed in this GP system through multiple program representations. Two program representations considered in this paper are the procedural representation (PR) and the tree representation (TR). Populations of these representations evolve separately. Individuals in each population migrate to the other and participate in evolution via representation change algorithms. Parallelism is achieved through use of the AutoMap/AutoLink MPI library. The differences in the locality properties of the representations serve as a source of new ideas for creating the final algorithm.

Genetic Programming (GP) is a form of nature-inspired computing, introduced over 30 years ago, with notable success in problems such as symbolic regression. However, there remains a lot of relatively unexploited potential for solving hard, real-world problems. There is consensus in the GP community that the lack of effective real-world benchmark problems negatively impacts the quality of research [4]. When a GP system is initialised, a number of parameters must be provided. The optimal setup configuration is often not known, due to the fact that many of the values are problem and domain specific, meaning the GP system is unable to produce satisfactory results. We believe that the implementation of continuous self-adaptation, along with the introduction of tunable and suitably difficult benchmark problems, will allow for the creation of more robust GP systems that are resilient to failure.

Transfer learning has attracted more attention in the machine-learning community recently. It aims to improve the learning performance on the domain of interest with the help of the knowledge acquired from a similar domain(s). However, there is only a limited number of research on tackling transfer learning in genetic programming for symbolic regression. This article attempts to fill this gap by proposing a new instance weighting framework for transfer learning in genetic programming-based symbolic regression. In the new framework, differential evolution is employed to search for optimal weights for source-domain instances, which helps genetic programming to identify more useful source-domain instances and learn from them. Meanwhile, a density estimation method is used to provide good starting points to help the search for the optimal weights while discarding some irrelevant or less important source-domain instances before learning regression models. The experimental results show that compared with genetic programming and support vector regression that learn only from the target instances, and learning from a mixture of instances from the source and target domains without any transfer learning component, the proposed method can evolve regression models which not only achieve notably better cross-domain generalization performance in stability but also reduce the trend of overfitting effectively. Meanwhile, these models are generally much simpler than those generated by the other GP methods.

Annotating automatic target recognition (ATR) is a highly challenging task, primarily due to the unavailability of labeled data in the target domain. Hence, it is essential to construct an optimal target domain classifier by utilizing the labeled information of the source domain images. The transductive transfer learning (TTL) method that incorporates a CycleGAN-based unpaired domain translation network has been previously proposed in the literature for effective ATR annotation. Although this method demonstrates great potential for ATR, it severely suffers from lower annotation performance, higher Fréchet Inception Distance (FID) score, and the presence of visual artifacts in the synthetic images. To address these issues, we propose a hybrid contrastive learning base unpaired domain translation (H-CUT) network that achieves a significantly lower FID score. It incorporates both attention and entropy to emphasize the domain-specific region, a noisy feature mixup module to generate high variational synthetic negative patches, and a modulated noise contrastive estimation (MoNCE) loss to reweight all negative patches using optimal transport for better performance. Our proposed contrastive learning and cycle-consistency-based TTL (C3TTL) framework consists of two H-CUT networks and two classifiers. It simultaneously optimizes cycle-consistency, MoNCE, and identity losses. In C3TTL, two H-CUT networks have been employed through a bijection mapping to feed the reconstructed source domain images into a pretrained classifier to guide the optimal target domain classifier. Extensive experimental analysis conducted on six ATR datasets demonstrates that the proposed C3TTL method is effective in annotating civilian and military vehicles, ships, planes, and human targets.

In this work, genetic programming systems are viewed as cooperative processes: individuals in the population cooperate to evolve a global solution. There are two basic forms of cooperation for distributed problem solving: task-sharing and result-sharing. We introduce two models that enable cooperation in genetic programming systems. The first model is based on task-sharing and the second one is based on result-sharing. We examine the effects of both forms of cooperation on the performance of genetic programming systems solving the insect locomotion problem. This article demostrates that cooperative genetic programming can be used to evolve several commonly observed gaits in insects.

<p>One of the greater issues in Genetic Programming (GP) is the computational effort required to run the evolution and discover a good solution. Phenomena such as program bloating (where genetic programs rapidly grow in size) can quickly exhaust available memory resources and slow down the evolutionary process, while the heavy cost of performing fitness evaluation can make problems which have a lot of available data very slow to solve. These issues may limit GP in some tasks it can appropriately be applied to, as well as inhibit its applications in time/space sensitive environments. In this thesis, we look at developing solutions to some of these issues in GP computational cost. First, we develop an algebraic program simplification method based on simple rules and hashing techniques, and use this method in conjunction with the standard GP on a variety of tasks. Our results suggest that program simplification can lead to a significant reduction in program size, while not significantly changing the effectiveness of the systems in finding solution programs. Secondly, we analyse the effects of program simplification on the internal GP "building blocks" to investigate whether simplification is a destructive or constructive force. Using two models for building blocks (numerical-nodes and the more complex fixed-depth subtree), we track building blocks through GP runs on a symbolic regression problem, both with and without using simplification. We find that the program simplification process can both disrupt and construct building blocks in the GP populations. However, GP systems using simplification appear to retain important building blocks, and the simplification process appears to lead to an increase in genetic diversity. These may help explain why using simplification does not reduce the effectiveness of GP systems in solving tasks. Lastly, we develop two methods of reducing the cost of fitness evaluation by reducing the number of node evaluations performed. The first method is elitism avoidance, which avoids re-evaluating programs which have been placed in the population using elitismreproduction. Thismethod reduces the CPU time for evolving solutions for six different GP tasks. The second method is a subtree caching mechanism which store fitness evaluations for subtrees in a cache so that they may be fetched when these subtrees are encountered in future fitness evaluations. Results suggest that using this mechanism can significantly reduce both the number of node evaluations and the overall CPU time used in evolving solutions, without reducing the fitness of the solutions produced.</p>

Detecting Electrocardiogram (ECG) abnormalities is the process of identifying irregular cardiac activities which may lead to severe heart damage or even sudden death. Due to the rapid development of cyberphysic systems and health informatics, embedding the function of ECG abnormality detection to various devices for real time monitoring has attracted more and more interest in the past few years. The existing machine learning and pattern recognition techniques developed for this purpose usually require sufficient labeled training data for each user. However, obtaining such supervised information is difficult, which makes the proposed ECG monitoring function unrealistic. To tackle the problem, we take advantage of existing well labeled ECG signals and propose a transductive transfer learning framework for the detection of abnormalities in ECG. In our model, unsupervised signals from target users are classified with knowledge transferred from the supervised source signals. In the experimental evaluation, we implemented our method on the MIT-BIH Arrhythmias Dataset and compared it with both anomaly detection and transductive learning baseline approaches. Extensive experiments show that our proposed algorithm remarkably outperforms all the compared methods, proving the effectiveness of it in detecting ECG abnormalities.

Unsupervised domain adaptation aims to transfer knowledge from labeled source domain to unlabeled target domain. Recently, multisource domain adaptation (MDA) has begun to attract attention. Its performance should go beyond simply mixing all source domains together for knowledge transfer. In this article, we propose a novel prototype-based method for MDA. Specifically, for solving the problem that the target domain has no label, we use the prototype to transfer the semantic category information from source domains to target domain. First, a feature extraction network is applied to both source and target domains to obtain the extracted features from which the domain-invariant features and domain-specific features will be disentangled. Then, based on these two kinds of features, the named inherent class prototypes and domain prototypes are estimated, respectively. Then a prototype mapping to the extracted feature space is learned in the feature reconstruction process. Thus, the class prototypes for all source and target domains can be constructed in the extracted feature space based on the previous domain prototypes and inherent class prototypes. By forcing the extracted features are close to the corresponding class prototypes for all domains, the feature extraction network is progressively adjusted. In the end, the inherent class prototypes are used as a classifier in the target domain. Our contribution is that through the inherent class prototypes and domain prototypes, the semantic category information from source domains is transformed into the target domain by constructing the corresponding class prototypes. In our method, all source and target domains are aligned twice at the feature level for better domain-invariant features and more closer features to the class prototypes, respectively. Several experiments on public data sets also prove the effectiveness of our method.

Genetic programming (GP) is applied to the design of fuzzy logic controllers (FLCs) for mobile robot path tracking. GP is applied to automatic discovery of full knowledge bases for use in fuzzy logic control applications. An extension to a rule learning GP system is presented that achieves this objective. In addition, GP is employed to handle selection of fuzzy set intersection operators (t-norms). The new GP system is applied to design a mobile robot path tracking controller and performance is shown to be comparable to that of a manually designed controller. GP was successfully applied to discover FLCs capable of steering a mobile robot to track straight-line paths in the plane. Instances of simultaneous evolution of membership functions and rules showed that GP was capable of evolving a FLC that demonstrated satisfactory responsiveness to various initial conditions while utilizing minimal human interface.

Programming languages provide a variety of mechanisms to associate names with values, and these mechanisms play a central role in programming practice. For example, they allow multiple references to the same storage location or function in different parts of a complex program. By contrast, the representations used in current genetic programming systems provide few if any naming mechanisms, and it is therefore generally not possible for evolved programs to use names in sophisticated ways. In this chapter we describe a new approach to names in genetic programming that is based on Holland’s concept of tags. We demonstrate the use of tag-based names, we describe some of the ways in which they may help to extend the power and reach of genetic programming systems, and we look at the ways that tag-based names are actually used in an evolved program that solves a robot navigation problem.Keywordsgenetic programmingmodularitynamestagsstack-based genetic programmingPush programming languagePushGP