A gaussian filter-based feature learning approach using genetic programming to image classification

Accurate estimation of river flow can have a significant importance in water resources management. In this study, Genetic programming (GP) and Support Vector Machine (SVM) methods were used to forecast daily discharge of Barandoozchay River. The daily discharge data of Barandoozchay River measured at the Dizaj hydrometric station during 2007 to 2011 was used for modeling, which 80% of the data used for training and remaining 20% used for testing of models. The results showed that in the both of considered methods, the models including discharges of one, two and three days ago had higher accuracy in verification step and the accuracy of models decreased with increasing discharge values. Comparing the performance of GP and SVM methods indicated that, however the accuracy of the GP method with the R=0.978 and RMSE=1.66 (m3/s) was slightly more than SVM method with R=0.976 and RMSE=1.80 (m3/s), but the SVM is easier than GP method. Thus, the SVM method can be used as an alternative method in forecasting daily river discharge.

The genetic programming (GP) method is proposed as a new approach to perform texture classification based directly on raw pixel data. Two alternative genetic programming representations are used to perform classification. These are dynamic range selection (DRS) and static range selection (SRS). This preliminary study uses four brodatz textures to investigate the applicability of the genetic programming method for binary texture classifications and multi-texture classifications.Results indicate that the genetic programming method, based directly on raw pixel data, is able to accurately classify different textures. The results show that the DRS method is well suited to the task of texture classification. The classifiers generated in our experiments by DRS have good performance over a variety of texture data and offer GP as a promising alternative approach for the difficult problem of texture classification.KeywordsGenetic ProgrammingClass LabelTexture ClassificationTraining AccuracyDivision PointThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Genetic Programming (GP) has been widely employed to create dispatching rules intelligently for production scheduling. The success of GP depends on a suitable terminal set of selected features. Specifically, techniques that consider feature selection in GP to enhance rule understandability for dynamic job shop scheduling (DJSS) have been successful. However, existing feature selection algorithms in GP focus more emphasis on obtaining more compact rules with fewer features than on improving effectiveness. This paper is an attempt at combining a novel GP method, GP via dynamic diversity management, with feature selection to design effective and interpretable dispatching rules for DJSS. The idea of the novel GP method is to achieve a progressive transition from exploration to exploitation by relating the level of population diversity to the stopping criteria and elapsed duration. We hypothesize that diverse and promising individuals obtained from the novel GP method can guide the feature selection to design competitive rules. The proposed approach is compared with three GP-based algorithms and 20 benchmark rules in the different job shop conditions and scheduling objectives. Experiments show that the proposed approach greatly outperforms the compared methods in generating more interpretable and effective rules for the three objective functions. Overall, the average improvement over the best-evolved rules by the other three GP-based algorithms is 13.28%, 12.57%, and 15.62% in the mean tardiness (MT), mean flow time (MFT), and mean weighted tardiness (MWT) objective, respectively.

A context‐free genetic programming (GP) method is presented that simulated local scale daily extreme (maximum and minimum) temperatures based on large scale atmospheric variables. The method evolves simple and optimal models for downscaling daily temperature at a station. The advantage of the context‐free GP method is that both the variables and constants of the candidate models are optimized and consequently the selection of the optimal model. The method is applied to the Chute‐du‐Diable weather station in Northeastern Canada along with the National Center for Environmental Prediction (NCEP) reanalysis datasets. The performance of the GP based downscaling models is compared to benchmarks from a commonly used statistical downscaling model. The experiment results show that the models evolved by the GP are simpler and more efficient for downscaling daily extreme temperature than the common statistical method. The different model test results indicate that the GP approach significantly outperforms the statistical method for the downscaling of daily minimum temperature, while for the maximum temperature the two methods are almost equivalent. However, the GP method remains slightly more effective for maximum temperature downscaling than the statistical method.

Image classification is an important task that has a wide range of applications. Convolutional neural networks (CNNs) are a common approach that can achieve promising performance in image classification. However, using CNNs to address a problem requires in-depth knowledge about CNN architectures and how it relates to the problem domain. Genetic programming (GP) as an evolutionary computation method can used to reduce the amount of knowledge required to design a CNN for a given problem domain by automatically searching for the optimal architecture. This paper proposes a new algorithm named, GPCNN, which encodes graph-based CNN architectures as trees and uses genetic operators, i.e. mutation, crossover and selection, to find better architectures. A more flexible crossover, partial subtree crossover, is also proposed to improve the search performance. As an preliminary work, GPCNN did not manage to achieve better performance than the state-of-the-art methods due to the limit on computational resource, but it is able to achieve better results than the baseline methods. More importantly, the proposed tree-based graph representation of CNN allows CNN architecture of various shapes, which has a great potential for future work in evolutionary automatic neural architecture search.

Genetic programming (GP) is a well-known evolutionary computation technique, which has been successfully used to solve various problems, such as optimization, image analysis, and classification. Transfer learning is a type of machine learning approach that can be used to solve complex tasks. Transfer learning has been introduced to GP to solve complex Boolean and symbolic regression problems with some promise. However, the use of transfer learning with GP has not been investigated to address complex image classification tasks with noise and rotations, where GP cannot achieve satisfactory performance, but GP with transfer learning may improve the performance. In this paper, we propose a novel approach based on transfer learning and GP to solve complex image classification problems by extracting and reusing blocks of knowledge/information, which are automatically discovered from similar as well as different image classification tasks during the evolutionary process. The proposed approach is evaluated on three texture data sets and three office data sets of image classification benchmarks, and achieves better classification performance than the state-of-the-art image classification algorithm. Further analysis on the evolved solutions/trees shows that the proposed approach with transfer learning can successfully discover and reuse knowledge/information extracted from similar or different problems to improve its performance on complex image classification problems.

A genetic programming-based method for image classification with small training data

Visualizing and analyzing convolution neural networks with gradient information

This paper describes a domain independent approach to the use of neural networks (NNs) and genetic programming (GP) for object detection problems. Instead of using high level features for a particular task, this approach uses domain independent pixel statistics for object detection. The paper first compares an NN method and a GP method on four image data sets providing object detection problems of increasing difficulty. The results show that the GP method performs better than the NN method on these problems but still produces a large number of false alarms on the difficult problem and computation cost is still high. To deal with these problems, we develop a new method called GP-refine that uses a two stage learning process. The new GP method further improves object detection performance on the difficult detection task.

Image classification is a complex but important task especially in the areas of machine vision and image analysis such as remote sensing and face recognition. One of the challenges in image classification is finding an optimal set of features for a particular task because the choice of features has direct impact on the classification performance. However the goodness of a feature is highly problem dependent and often domain knowledge is required. To address these issues we introduce a Genetic Programming (GP) based image classification method, Two-Tier GP, which directly operates on raw pixels rather than features. The first tier in a classifier is for automatically defining features based on raw image input, while the second tier makes decision. Compared to conventional feature based image classification methods, Two-Tier GP achieved better accuracies on a range of different tasks. Furthermore by using the features defined by the first tier of these Two-Tier GP classifiers, conventional classification methods obtained higher accuracies than classifying on manually designed features. Analysis on evolved Two-Tier image classifiers shows that there are genuine features captured in the programs and the mechanism of achieving high accuracy can be revealed. The Two-Tier GP method has clear advantages in image classification, such as high accuracy, good interpretability and the removal of explicit feature extraction process. © 2012 IEEE.

Nonlinear model identification of an experimental ball-and-tube system using a genetic programming approach

One of the major challenges in proteomics is peptide identification from mass spectra containing high noise ratio and small number of signal (b-/y-ions) peaks. However, the accuracy and reliability of peptide identification in such highly imbalanced MS/MS data can be improved by applying a preprocessing step prior to peptide identification aiming at discriminating b-/y-ions from noise peaks in the spectra. In this study, we report a genetic programming (GP)-based preprocessing method for de-noising highly imbalanced and noisy CID MS/MS spectra. GP now becomes a popular machine learning method via automatic programming. GP preprocesses the highly noisy MS/MS spectra by classifying peaks as noise peaks or signal peaks in a binary classification manner. Meanwhile, a set of spectral fragment features based on the MS/MS fragmentation rules is extracted from the dataset to investigate their discriminating abilities by GP. A MS/MS spectral dataset containing thousands of spectra are used to train the GP model. As the GP tree-based representation has the capability for implicit feature selection during the evolutionary process, the evolved GP model with the selected features is compared with the best threshold-based method. The results show that the GP method improved the reliability of peptide identification and increased the identification rate of a de novo sequencing tool, PEAKS, to 99.4% from 80.1% achieved by the best threshold-based method. Moreover, the result of peptide identification by a database search tool, SEQUEST, using the data preprocessed by the GP method was statistically significant compared to the other methods.

Semantic genetic programming is a recent, rapidly growing trend in Genetic Programming (GP) that aims at opening the 'black box' of the evaluation function and make explicit use of more information on program behavior in the search. In the most common scenario of evaluating a GP program on a set of input-output examples (fitness cases), the semantic approach characterizes program with a vector of outputs rather than a single scalar value (fitness). The past research on semantic GP has demonstrated that the additional information obtained in this way facilitates designing more effective search operators. In particular, exploiting the geometric properties of the resulting semantic space leads to search operators with attractive properties, which have provably better theoretical characteristics than conventional GP operators. This in turn leads to dramatic improvements in experimental comparisons. The aim of the tutorial is to give a comprehensive overview of semantic methods in genetic programming, illustrate in an accessible way a formal geometric framework for program semantics to design provably good mutation and crossover operators for traditional GP problem domains, and to analyze rigorously their performance (runtime analysis). A number of real-world applications of this framework will be also presented. Other promising emerging approaches to semantics in GP will be reviewed. In particular, the recent developments in the behavioral programming, which aims at characterizing the entire program behavior (and not only program outputs) will be covered as well. Current challenges and future trends in semantic GP will be identified and discussed. Selected methods and concepts will be accompanied with live software demonstrations. Also, efficient implementation of semantic search operators may be challenging. We will illustrate very efficient, concise and elegant implementations of these operators, which are available for download from the web.

Feature extraction, as one essential step of image classification, can potentially reduce image data dimensionality and capture effective information for improving performance. However, most existing image descriptors are designed to conduct specific tasks and might not be sufficient for different types of images. Genetic programming (GP) can automatically extract multiple important and discriminative features by incorporating diverse image descriptors into a GP program. Furthermore, different regions in an image have different structural characteristics. In this paper, we propose a region adaptive image classification approach based on GP, which can automatically extract informative image features by automatically applying different image descriptors in different regions of an image. A new flexible GP program structure with a new function set and a new terminal set is developed in this approach. The performance of the proposed method is evaluated on four various data sets and compared with other state-of-the-art classification methods. Experimental results illustrate that the proposed approach is capable of achieving better or competitive performance than these baseline methods. Further analysis of some good programs shows the high interpretability of the proposed method.

The excellent feature extraction ability of deep convolutional neural networks (DCNNs) has been demonstrated in many image processing tasks, by which image classification can achieve high accuracy with only raw input images. However, the specific image features that influence the classification results are not readily determinable and what lies behind the predictions is unclear. This study proposes a method combining the Sobel and Canny operators and an Inception module for ship classification. The Sobel and Canny operators obtain enhanced edge features from the input images. A convolutional layer is replaced with the Inception module, which can automatically select the proper convolution kernel for ship objects in different image regions. The principle is that the high-level features abstracted by the DCNN, and the features obtained by multi-convolution concatenation of the Inception module must ultimately derive from the edge information of the preprocessing input images. This indicates that the classification results are based on the input edge features, which indirectly interpret the classification results to some extent. Experimental results show that the combination of the edge features and the Inception module improves DCNN ship classification performance. The original model with the raw dataset has an average accuracy of 88.72%, while when using enhanced edge features as input, it achieves the best performance of 90.54% among all models. The model that replaces the fifth convolutional layer with the Inception module has the best performance of 89.50%. It performs close to VGG-16 on the raw dataset and is significantly better than other deep neural networks. The results validate the functionality and feasibility of the idea posited.