Extracting image features for classification by two-tier genetic programming

Genetic programming (GP) is a well-known evolutionary computation technique, which has been successfully used to solve various problems, such as optimization, image analysis, and classification. Transfer learning is a type of machine learning approach that can be used to solve complex tasks. Transfer learning has been introduced to GP to solve complex Boolean and symbolic regression problems with some promise. However, the use of transfer learning with GP has not been investigated to address complex image classification tasks with noise and rotations, where GP cannot achieve satisfactory performance, but GP with transfer learning may improve the performance. In this paper, we propose a novel approach based on transfer learning and GP to solve complex image classification problems by extracting and reusing blocks of knowledge/information, which are automatically discovered from similar as well as different image classification tasks during the evolutionary process. The proposed approach is evaluated on three texture data sets and three office data sets of image classification benchmarks, and achieves better classification performance than the state-of-the-art image classification algorithm. Further analysis on the evolved solutions/trees shows that the proposed approach with transfer learning can successfully discover and reuse knowledge/information extracted from similar or different problems to improve its performance on complex image classification problems.

Feature extraction is essential for solving image classification by transforming low-level pixel values into high-level features. However, extracting effective features from images is challenging due to high variations across images in scale, rotation, illumination, and background. Existing methods often have a fixed model complexity and require domain expertise. Genetic programming (GP) with a flexible representation can find the best solution without the use of domain knowledge. This article proposes a new GP-based approach to automatically learning informative features for different image classification tasks. In the new approach, a number of image-related operators, including filters, pooling operators, and feature extraction methods, are employed as functions. A flexible program structure is developed to integrate different functions and terminals into a single tree/solution. The new approach can evolve solutions of variable depths to extract various numbers and types of features from the images. The new approach is examined on 12 different image classification tasks of varying difficulty and compared with a large number of effective algorithms. The results show that the new approach achieves better classification performance than most benchmark methods. The analysis of the evolved programs/solutions and the visualization of the learned features provide deep insights on the proposed approach.

Although the existing traditional image classification methods have been widely applied in practical problems, there are some problems in the application process, such as unsatisfactory effects, low classification accuracy, and weak adaptive ability. This method separates image feature extraction and classification into two steps for classification operation. The deep learning model has a powerful learning ability, which integrates the feature extraction and classification process into a whole to complete the image classification test, which can effectively improve the image classification accuracy. However, this method has the following problems in the application process: first, it is impossible to effectively approximate the complex functions in the deep learning model. Second, the deep learning model comes with a low classifier with low accuracy. So, this paper introduces the idea of sparse representation into the architecture of the deep learning network and comprehensively utilizes the sparse representation of well multidimensional data linear decomposition ability and the deep structural advantages of multilayer nonlinear mapping to complete the complex function approximation in the deep learning model. And a sparse representation classification method based on the optimized kernel function is proposed to replace the classifier in the deep learning model, thereby improving the image classification effect. Therefore, this paper proposes an image classification algorithm based on the stacked sparse coding depth learning model-optimized kernel function nonnegative sparse representation. The experimental results show that the proposed method not only has a higher average accuracy than other mainstream methods but also can be good adapted to various image databases. Compared with other deep learning methods, it can better solve the problems of complex function approximation and poor classifier effect, thus further improving image classification accuracy.

Image classification is getting more attention in the area of computer vision. During the past few years, a lot of research has been done on image classification using classical machine learning and deep learning techniques. Presently, deep learning-based techniques have given stupendous results. The performance of a classification system depends on the quality of features extracted from an image. The better is the quality of extracted features, the more the accuracy will be. Although, numerous deep learning-based methods have shown enormous performance in image classification, still due to various challenges deep learning methods are not able to extract all the important information from the image. This results in a reduction in overall classification accuracy. The goal of the present research is to improve the image classification performance by combining the deep features extracted using popular deep convolutional neural network, VGG19, and various handcrafted feature extraction methods, i.e., SIFT, SURF, ORB, and Shi-Tomasi corner detector algorithm. Further, the extracted features from these methods are classified using various machine learning classification methods, i.e., Gaussian Naïve Bayes, Decision Tree, Random Forest, and eXtreme Gradient Boosting (XGBClassifier) classifier. The experiment is carried out on a benchmark dataset Caltech-101. The experimental results indicate that Random Forest using the combined features give 93.73% accuracy and outperforms other classifiers and methods proposed by other authors. The paper concludes that a single feature extractor whether shallow or deep is not enough to achieve satisfactory results. So, a combined approach using deep learning features and traditional handcrafted features is better for image classification.

Recent research has shown the initial success of sparse coding (Sc) in solving many computer vision tasks. Motivated by the fact that kernel trick can capture the nonlinear similarity of features, which helps in finding a sparse representation of nonlinear features, we propose kernel sparse representation (KSR). Essentially, KSR is a sparse coding technique in a high dimensional feature space mapped by an implicit mapping function. We apply KSR to feature coding in image classification, face recognition, and kernel matrix approximation. More specifically, by incorporating KSR into spatial pyramid matching (SPM), we develop KSRSPM, which achieves a good performance for image classification. Moreover, KSR-based feature coding can be shown as a generalization of efficient match kernel and an extension of Sc-based SPM. We further show that our proposed KSR using a histogram intersection kernel (HIK) can be considered a soft assignment extension of HIK-based feature quantization in the feature coding process. Besides feature coding, comparing with sparse coding, KSR can learn more discriminative sparse codes and achieve higher accuracy for face recognition. Moreover, KSR can also be applied to kernel matrix approximation in large scale learning tasks, and it demonstrates its robustness to kernel matrix approximation, especially when a small fraction of the data is used. Extensive experimental results demonstrate promising results of KSR in image classification, face recognition, and kernel matrix approximation. All these applications prove the effectiveness of KSR in computer vision and machine learning tasks.

Rich heterogeneous data provided by social networks can be very big, which imposes considerable challenges for object extraction and image classification. Therefore, the objective of this work is to propose an unsupervised object cosegmentation method that could be notably efficient to improve image classification performance. The main goal of cosegmentation is to extract the salient common objects within each image. To this end, we propose to minimize an energy function based on the Markov Random Field using the saliency detection, while considering linear dependence of generated foreground histograms of the input image collection. In fact, the saliency detection is processed in two steps. In each image, we detect salient objects, by considering appearance similarity and spatial distributions of image pixels. Then, fuzzy quantification is used to correct the belonging of pixels to the foreground objects. Finally, an iterative optimization permits to enhance the final segmentation results. The proposed method has been validated as a preprocessing step for image classification. Indeed, to enhance cosegmentation-based classification performance, we have applied a semi-supervised object classification based on ensemble projection. Qualitative and quantitative evaluations of the proposed cosegmentation and classification techniques on the iCoseg, CDS and Oxford Flowers 17 datasets demonstrate the effectiveness of the proposed framework.

Exploitation of the spectral capabilities of modern hyperspectral image demands efficient preprocessing and analyses methods. Analysts' choice of classifier and dimensionality reduction (DR) method and the harmony between them determine the accuracy of image classification. Multiple classifier system (MCS) has the potential to combine the relative advantages of several classifiers into a single image classification exercise for the hyperspectral image classification. In this paper, we propose an algorithmic extension of the MCS, named as dynamic classifier system (DCS), which exploits the context-based image and information class characteristics represented by multiple DR methods for hyperspectral image classification for land cover mapping. The proposed DCS algorithm pairs up optimal combinations of classifiers and DR methods specific to the hyperspectral image and performs image classifications based only on the identified combinations. Further, the impact of various trainable and nontrainable combination functions on the performance of the proposed DCS has been assessed. Image classifications were carried out on five multi-site airborne hyperspectral images using the proposed DCS and were compared with the MCS and SVM based supervised image classifications with and without DR. The results indicate the potential of the proposed DCS algorithm to increase the classification accuracy considerably over that of MCS or SVM supervised image classifications.

Deep CNN models have become state-of-the-art techniques in many application, e.g., face recognition, speaker recognition, and image classification. Although many studies address on speedup or compression of individual models, very few studies focus on co-compressing and unifying models from different modalities. In this work, to joint and compress face and speaker recognition models, a shared-codebook approach is adopted to reduce the redundancy of the combined model. Despite the modality of the inputs of these two CNN models are quite different, the shared codebook can support two CNN models of sound and image for speaker and face recognition. Experiments show the promising results of unified and co-compressing heterogeneous models for efficient inference.

© Springer Nature Switzerland AG 2018. To learn image features automatically from the problems being tackled is more effective for classification. However, it is very difficult due to image variations and the high dimensionality of image data. This paper proposes a new feature learning approach based on Gaussian filters and genetic programming (GauGP) for image classification. Genetic programming (GP) is a well-known evolutionary learning technique and has been applied to many visual tasks, showing good learning ability and interpretability. In the proposed GauGP method, a new program structure, a new function set and a new terminal set are developed, which allow it to detect small regions from the input image and to learn discriminative features using Gaussian filters for image classification. The performance of GauGP is examined on six different data sets of varying difficulty and compared with four GP methods, eight traditional approaches and convolutional neural networks. The experimental results show GauGP achieves significantly better or similar performance in most cases.

Extracting image feature points and classification methods is the key of content-based image classification. In this paper, SIFT(Scale-invariant feature transform) algorithm is used to extract feature points, all feature points extracted are clustered by K-means clustering algorithm, and then BOW(bag of word) of each image is constructed. Finally, SVM(Support Vector Machine) is used to train a multi-class classifier to classify images. The SIFT algorithm has a strong tolerance for scaling, rotation, brightness changes, and noise. The k-means algorithm is simple in structure and fast in convergence. SVM can get better results in small sample training set and has excellent generalization ability. Experimental results show that the accuracy of image classification in the above methods is about 90%.

Feature extraction, as one essential step of image classification, can potentially reduce image data dimensionality and capture effective information for improving performance. However, most existing image descriptors are designed to conduct specific tasks and might not be sufficient for different types of images. Genetic programming (GP) can automatically extract multiple important and discriminative features by incorporating diverse image descriptors into a GP program. Furthermore, different regions in an image have different structural characteristics. In this paper, we propose a region adaptive image classification approach based on GP, which can automatically extract informative image features by automatically applying different image descriptors in different regions of an image. A new flexible GP program structure with a new function set and a new terminal set is developed in this approach. The performance of the proposed method is evaluated on four various data sets and compared with other state-of-the-art classification methods. Experimental results illustrate that the proposed approach is capable of achieving better or competitive performance than these baseline methods. Further analysis of some good programs shows the high interpretability of the proposed method.

Image classification is an important task that has a wide range of applications. Convolutional neural networks (CNNs) are a common approach that can achieve promising performance in image classification. However, using CNNs to address a problem requires in-depth knowledge about CNN architectures and how it relates to the problem domain. Genetic programming (GP) as an evolutionary computation method can used to reduce the amount of knowledge required to design a CNN for a given problem domain by automatically searching for the optimal architecture. This paper proposes a new algorithm named, GPCNN, which encodes graph-based CNN architectures as trees and uses genetic operators, i.e. mutation, crossover and selection, to find better architectures. A more flexible crossover, partial subtree crossover, is also proposed to improve the search performance. As an preliminary work, GPCNN did not manage to achieve better performance than the state-of-the-art methods due to the limit on computational resource, but it is able to achieve better results than the baseline methods. More importantly, the proposed tree-based graph representation of CNN allows CNN architecture of various shapes, which has a great potential for future work in evolutionary automatic neural architecture search.

Sea ice disasters are already one of the most serious marine disasters in the Bohai Sea region of our country, which have seriously affected the coastal economic development and residents’ lives. Sea ice classification is an important part of sea ice detection. Hyperspectral imagery and multispectral imagery contain rich spectral information and spatial information and provide important data support for sea ice classification. At present, most sea ice classification methods mainly focus on shallow learning based on spectral features, and the good performance of the deep learning method in remote sensing image classification provides a new idea for sea ice classification. However, the level of deep learning is limited due to the influence of input size in sea ice image classification, and the deep features in the image cannot be fully mined, which affects the further improvement of sea ice classification accuracy. Therefore, this paper proposes an image classification method based on multilevel feature fusion using residual network. First, the PCA method is used to extract the first principal component of the original image, and the residual network is used to deepen the number of network layers. The FPN, PAN, and SPP modules increase the mining between layer and layer features and merge the features between different layers to further improve the accuracy of sea ice classification. In order to verify the effectiveness of the method in this paper, sea ice classification experiments were performed on the hyperspectral image of Bohai Bay in 2008 and the multispectral image of Bohai Bay in 2020. The experimental results show that compared with the algorithm with fewer layers of deep learning network, the method proposed in this paper utilizes the idea of residual network to deepen the number of network layers and carries out multilevel feature fusion through FPN, PAN, and SPP modules, which effectively solves the problem of insufficient deep feature extraction and obtains better classification performance.

The feature extraction-coding-pooling framework improves the performance of image classification because it generates robust and discriminative image representation. The proposed system uses saliency driven multi-scale nonlinear diffusion filtering with linear distance coding (LDC) method for image classification. The saliency driven multi-scale nonlinear diffusion filtering generates the images at small, mid and high scale and concatenation of information at these scales produces robust image classification using LDC. The class manifolds generated by this classification system are given as input to image retrieval system which finally retrieves all images those are relevant to supplied image.

In recent years, content-based image retrieval method has attracted the attention of many researchers. With the emergence of large-scale computer vision data set, large-scale image classification is going to become an important challenge in the field of computer vision. The existing image classification algorithms mainly have the following defects: (1) lack of sufficient theoretical basis; (2) lower classification accuracy and larger error; (3) algorithm running time is too long because of large amount of data. In order to solve these problems, we introduce an algorithm based on visual saliency and feature extraction theory. Visual saliency means that intelligent algorithm can mark the significant areas in the picture through the simulation of human visual characteristics, and we can preliminarily screen the large-scale date set in terms of saliency extraction, in order to constitute an initial categories set. Image classification mainly includes three processes, that are feature extraction, computation of image pattern vector, classification. Feature extraction is the key factor that affects the performance of image classification. Comparing to global features of image with blur background, partial occlusion, and illumination changes, local features are mode adaptive. Traditional image classification algorithms only use single feature, thus these methods bring limitations. In this paper, we have a thorough research of image classification methods based on feature combination, and we also propose an algorithm based on multiple feature. At the same time, in order to solve the problem that traditional methods dealing with the whole image in a non-hierarchical way, we introduce the theory of visual saliency detection. The results obviously shows that efficient feature extraction combined with the theory of visual saliency are significant to image classification.