A Region Adaptive Image Classification Approach Using Genetic Programming

Transductive transfer learning based Genetic Programming for balanced and unbalanced document classification using different types of features

© Springer Nature Switzerland AG 2018. To learn image features automatically from the problems being tackled is more effective for classification. However, it is very difficult due to image variations and the high dimensionality of image data. This paper proposes a new feature learning approach based on Gaussian filters and genetic programming (GauGP) for image classification. Genetic programming (GP) is a well-known evolutionary learning technique and has been applied to many visual tasks, showing good learning ability and interpretability. In the proposed GauGP method, a new program structure, a new function set and a new terminal set are developed, which allow it to detect small regions from the input image and to learn discriminative features using Gaussian filters for image classification. The performance of GauGP is examined on six different data sets of varying difficulty and compared with four GP methods, eight traditional approaches and convolutional neural networks. The experimental results show GauGP achieves significantly better or similar performance in most cases.

The goodness of the features extracted from the instances and the number of training instances are two key components in machine learning, and building an effective model is largely affected by these two factors. Acquiring a large number of training instances is very expensive in some situations such as in the medical domain. Designing a good feature set, on the other hand, is very hard and often requires domain expertise. In computer vision, image descriptors have emerged to automate feature detection and extraction; however, domain-expert intervention is typically needed to develop these descriptors. The aim of this paper is to utilize genetic programming to automatically construct a rotation-invariant image descriptor by synthesizing a set of formulas using simple arithmetic operators and first-order statistics, and determining the length of the feature vector simultaneously using only two instances per class. Using seven texture classification image datasets, the performance of the proposed method is evaluated and compared against eight domain-expert hand-crafted image descriptors. Quantitatively, the proposed method has significantly outperformed, or achieved comparable performance to, the competitor methods. Qualitatively, the analysis shows that the descriptors evolved by the proposed method can be interpreted.

1089-778X © 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. The goodness of the features extracted from the instances and the number of training instances are two key components in machine learning, and building an effective model is largely affected by these two factors. Acquiring a large number of training instances is very expensive in some situations such as in the medical domain. Designing a good feature set, on the other hand, is very hard and often requires domain expertise. In computer vision, image descriptors have emerged to automate feature detection and extraction; however, domain-expert intervention is typically needed to develop these descriptors. The aim of this paper is to utilize genetic programming to automatically construct a rotation-invariant image descriptor by synthesizing a set of formulas using simple arithmetic operators and first-order statistics, and determining the length of the feature vector simultaneously using only two instances per class. Using seven texture classification image datasets, the performance of the proposed method is evaluated and compared against eight domain-expert hand-crafted image descriptors. Quantitatively, the proposed method has significantly outperformed, or achieved comparable performance to, the competitor methods. Qualitatively, the analysis shows that the descriptors evolved by the proposed method can be interpreted.

Genetic programming (GP) is a well-known evolutionary computation technique, which has been successfully used to solve various problems, such as optimization, image analysis, and classification. Transfer learning is a type of machine learning approach that can be used to solve complex tasks. Transfer learning has been introduced to GP to solve complex Boolean and symbolic regression problems with some promise. However, the use of transfer learning with GP has not been investigated to address complex image classification tasks with noise and rotations, where GP cannot achieve satisfactory performance, but GP with transfer learning may improve the performance. In this paper, we propose a novel approach based on transfer learning and GP to solve complex image classification problems by extracting and reusing blocks of knowledge/information, which are automatically discovered from similar as well as different image classification tasks during the evolutionary process. The proposed approach is evaluated on three texture data sets and three office data sets of image classification benchmarks, and achieves better classification performance than the state-of-the-art image classification algorithm. Further analysis on the evolved solutions/trees shows that the proposed approach with transfer learning can successfully discover and reuse knowledge/information extracted from similar or different problems to improve its performance on complex image classification problems.

Image classification is an important task that has a wide range of applications. Convolutional neural networks (CNNs) are a common approach that can achieve promising performance in image classification. However, using CNNs to address a problem requires in-depth knowledge about CNN architectures and how it relates to the problem domain. Genetic programming (GP) as an evolutionary computation method can used to reduce the amount of knowledge required to design a CNN for a given problem domain by automatically searching for the optimal architecture. This paper proposes a new algorithm named, GPCNN, which encodes graph-based CNN architectures as trees and uses genetic operators, i.e. mutation, crossover and selection, to find better architectures. A more flexible crossover, partial subtree crossover, is also proposed to improve the search performance. As an preliminary work, GPCNN did not manage to achieve better performance than the state-of-the-art methods due to the limit on computational resource, but it is able to achieve better results than the baseline methods. More importantly, the proposed tree-based graph representation of CNN allows CNN architecture of various shapes, which has a great potential for future work in evolutionary automatic neural architecture search.

Image classification is a complex but important task especially in the areas of machine vision and image analysis such as remote sensing and face recognition. One of the challenges in image classification is finding an optimal set of features for a particular task because the choice of features has direct impact on the classification performance. However the goodness of a feature is highly problem dependent and often domain knowledge is required. To address these issues we introduce a Genetic Programming (GP) based image classification method, Two-Tier GP, which directly operates on raw pixels rather than features. The first tier in a classifier is for automatically defining features based on raw image input, while the second tier makes decision. Compared to conventional feature based image classification methods, Two-Tier GP achieved better accuracies on a range of different tasks. Furthermore by using the features defined by the first tier of these Two-Tier GP classifiers, conventional classification methods obtained higher accuracies than classifying on manually designed features. Analysis on evolved Two-Tier image classifiers shows that there are genuine features captured in the programs and the mechanism of achieving high accuracy can be revealed. The Two-Tier GP method has clear advantages in image classification, such as high accuracy, good interpretability and the removal of explicit feature extraction process. © 2012 IEEE.

This study aims at developing an intelligent agent that can recognize user-specific emotions and can self-evolve. Previous studies have explored several methods to develop the model and improve the results while maintaining the feasibility of real-time implementation for later stages. We evolved the emotion recognition module by using Genetic Programming (GP) and explored several optimizations. We investigated and compared the evolution of a unique classifier (evolved from data from a single specific subject only), the evolution of a general classifier (evolved from data from multiple subjects), and the evolution of an adaptive classifier by implementing incremental GP (evolved incrementally, first from multiple subjects and then from a single specific subject). We conducted the experiments by using the same budget in terms of evolution sessions to obtain the best programs for a fair comparison between general approach, user-specific approach, and adaptive approach. We then performed repeated experiments to verify the robustness of the method. From the results, we concluded that, on an average, adaptive approach not only resulted in faster evolution time, but also achieved better accuracy in emotion recognition.

The mutation operator is the only source of variation in Evolutionary Programming. In the past these have been human nominated and included the Gaussian, Cauchy, and the Levy distributions. We automatically design mutation operators (probability distributions) using Genetic Programming. This is done by using a standard Gaussian random number generator as the terminal set and and basic arithmetic operators as the function set. In other words, an arbitrary random number generator is a function of a randomly (Gaussian) generated number passed through an arbitrary function generated by Genetic Programming. Rather than engaging in the futile attempt to develop mutation operators for arbitrary benchmark functions (which is a consequence of the No Free Lunch theorems), we consider tailoring mutation operators for particular function classes. We draw functions from a function class (a probability distribution over a set of functions). The mutation probability distribution is trained on a set of function instances drawn from a given function class. It is then tested on a separate independent test set of function instances to confirm that the evolved probability distribution has indeed generalized to the function class. Initial results are highly encouraging: on each of the ten function classes the probability distributions generated using Genetic Programming outperform both the Gaussian and Cauchy distributions.

The Identification of rock constitutive model is a typical nonlinear system problem; introduction of the intelligent methods has greatly stimulated the research. In this paper, genetic algorithm , BP neural network and genetic programming are used in the identification of constitutive model ,the capacity of three intelligent methods in the model identification is compared ,whose results show that genetic algorithm can largely reduce the probability of trapping into localized optimum solution by means of its global searching ability ,however , because its pre-assumption of constitutive model has already lead to some error ,its fitting results are not so good . BP neural network and genetic programming need not pre-assume the structure of constitutive model ,the nonlinear mapping expressions between stress and strain are obtained by self-organization and self-learning .But ,the determination of BP neural network structure is experience-depending and time-consuming . In addition, genetic programming need not define its structure, so long as the functional set and terminal set are chosen, after series of genetic operations, the nonlinear relations between variables are outputted, therefore, the operation of genetic programming is simple which realize the visualization of the constitutive equations.

The main indicators of the performance of an image classifier include classification accuracy, classification efficiency and evolution efficiency, whereas the sizes of the programs involved in genetic programming stand as one of the major factors that influence the performance of the classifier. Some effective means are introduced in this paper for the control of program sizes, which is done through scientific construction of fitness functions of the classifier in image classification in genetic programming. Tests showed that, the growth of program sizes were effectively put under control during the course of evolution, which, in turn, greatly improved the general performance of the image classifier in genetic programming.

Genetic programming has been explored in recent works to evolve hyper-heuristics for dynamic flexible job shop scheduling. To generate optimum rules, the algorithm searches a space of trees composed from a set of terminals and operators. Since the search space is exponentially proportional to the size of the terminal set, it is preferred to opt out any insignificant terminals. Feature selection techniques has been employed to reduce the terminal set size without discarding any important information and they have proven to be effective for enhancing search performance and efficiency for dynamic flexible job shop scheduling. In this paper, we extends our previous work by adding a modified version of the two-stage genetic programming algorithm and by comparing the different methods in a larger experimental setup. The results show that feature selection can generate better rules in most of the cases while also being more efficient to in a production environment.KeywordsFeature selectionFlexible job shop schedulingDynamic schedulingGenetic programmingHyper heuristics

MPEG-7 standard describes visual descriptors and performance metrics for image classification. Amongst all image features, color and texture are more visually expressive and hence are attractive for visual descriptors. Further, combination of features makes image classification more relevant and robust. This paper proposes efficient methods for image classification using 3 MPEG-7 descriptors to represent color and texture features. Additionally, we apply Support Vector Machine and Fuzzy algorithm to extract linguistic fuzzy rules, in order to bridge the gap between the lower-lever descriptors and the high-level semantic of an image. The Support Vector Machine Fuzzy Network was evaluated using images from the aceMedia Repository and more specifically in a beach/urban scenes classification problem.

Melanoma is the deadliest form of skin cancer that causes around 75% of deaths worldwide. However, most of the skin cancers can be cured, especially if detected and treated early. Existing approaches have employed various feature extraction methods, where different types of features are used individually for skin image classification which may not provide sufficient information to the classification algorithm necessary to discriminate between classes, leading to sub-optimal performance. This study develops a novel skin image classification method using multi-tree genetic programming (GP). To capture local information from gray and color skin images, Local Binary Pattern is used in this work. In addition, for capturing global information, variation in color within the lesion and the skin regions, and domain-specific lesion border shape features are extracted. GP with a multi-tree representation is employed to use multiple types of features. Genetic operators such as crossover and mutation are designed accordingly in order to select a single type of features at terminals in one tree of the GP individual. The performance of the proposed method is assessed using two skin image datasets having images captured from multiple modalities, and compared with six most commonly used classification algorithms as well as the standard (single-tree) wrapper and embedded GP methods. The results show that the proposed method has significantly outperformed all these classification methods. Being interpretable and fast in terms of the computation time, this method can help dermatologist identify prominent skin image features, specific to a type of skin cancer in real-time situations.

In this study we use Genetic Programming (GP) as an offline hyper-heuristic to evolve a mutation operator for Evolutionary Programming. This is done using the Gaussian and uniform distributions as the terminal set, and arithmetic operators as the function set. The mutation operators are automatically designed for a specific function class. The contribution of this paper is to show that a GP can not only automatically design a mutation operator for Evolutionary Programming (EP) on functions generated from a specific function class, but also can design more general mutation operators on functions generated from groups of function classes. In addition, the automatically designed mutation operators also show good performance on new functions generated from a specific function class or a group of function classes.