Enhancing the diversity of genetic algorithm for improved feature selection

Focusing on the problem of premature convergence existed in genetic algorithm for assembly sequence planning, the reasons were analyzed from genetic operations. It was pointed out that keeping population diversity and gene schema were the keys to avoid premature convergence. Checking up chromosomes and regulating them with the same mechanism will alter genetic information or destroy schema structure. They will also aggravate premature convergence. The influences of variation probability on population diversity and evolution progress were discussed in detail. To alleviate premature convergence, an extend operator was presented. It can enhance population diversity and local search ability by replacing the bad individuals with the son individuals produced by good individuals dithering. Supplantation operator was adopted to improve the evolution ability. It will check up difference between chromosomes of individuals and add a punish term to the little difference individual at initial evolutions. In order to prevent loss of available gene, gene repair strategy was presented to ill genes. Simulation results for an assembly sequence planning show validity of the presented algorithm.

In order to obtain the higher classification accuracy in specific categories for the different feature subset, a hierarchical classification algorithm based on Feature Selection is proposed, and is used for synthetic aperture radar (SAR) image classification, and feature selection is achieved by Genetic algorithm. The algorithm has two main characteristics: one is hierarchical classification which consists of many two-class classifier, and the two-class classifier is trained by the optimal feature subset which is selected according to different categories; the second is the classifier of support vector machine SVM (Support Vector Machine); the two is Genetic algorithm which can search out the optimal feature subset and parameters of support vector machine that is most suitable for the category, by unified coding the feature set and the parameters of SVM to constitute the Chromosome. The experiment on the first polarimetric SAR data show that the algorithm can obtain higher classification accuracy rate.

The search ability of an Evolutionary Algorithm (EA) depends on the variation among the individuals in the population [1, 2, 3]. Maintaining an optimal level of diversity in the EA population is imperative to ensure that progress of the EA search is unhindered by premature convergence to suboptimal solutions. Clearer understanding of the concept of population diversity, in the context of evolutionary search and premature convergence in particular, is the key to designing efficient EAs. To this end, this paper first presents a brief analysis of the EA population diversity issues. Next we present an investigation on a counter-niching EA technique [2] that introduces and maintains constructive diversity in the population. The proposed approach uses informed genetic operations to reach promising, but unexplored or under-explored areas of the search space, while discouraging premature local convergence. Simulation runs on a suite of standard benchmark test functions with Genetic Algorithm (GA) implementation shows promising results.

Computer-aided diagnosis has the potential of increasing diagnostic accuracy by providing a second reading to radiologists. In many computerized schemes, numerous features can be extracted to describe suspect image regions. A subset of these features is then employed in a data classifier to determine whether the suspect region is abnormal or normal. Different subsets of features will, in general, result in different classification performances. A feature selection method is often used to determine an "optimal" subset of features to use with a particular classifier. A classifier performance measure (such as the area under the receiver operating characteristic curve) must be incorporated into this feature selection process. With limited datasets, however, there is a distribution in the classifier performance measure for a given classifier and subset of features. In this paper, we investigate the variation in the selected subset of "optimal" features as compared with the true optimal subset of features caused by this distribution of classifier performance. We consider examples in which the probability that the optimal subset of features is selected can be analytically computed. We show the dependence of this probability on the dataset sample size, the total number of features from which to select, the number of features selected, and the performance of the true optimal subset. Once a subset of features has been selected, the parameters of the data classifier must be determined. We show that, with limited datasets and/or a large number of features from which to choose, bias is introduced if the classifier parameters are determined using the same data that were employed to select the "optimal" subset of features.

With the continuous development of Internet technology, data gradually present a complicated and high-dimensional trend. These high-dimensional data have a large number of redundant features and irrelevant features, which bring great challenges to the existing machine learning algorithms. Feature selection is one of the important research topics in the fields of machine learning, pattern recognition and data mining, and it is also an important means in the data preprocessing stage. Feature selection is to look for the optimal feature subset from the original feature set, which would improve the classification accuracy and reduce the machine learning time. The traditional feature selection algorithm tends to ignore the kind of feature which has a weak distinguishing capacity as a monomer, whereas the feature group’s distinguishing capacity is strong. Therefore, a new dynamic interaction feature selection (DIFS) algorithm is proposed in this paper. Initially, under the theoretical framework of interactive information, it redefines the relevance, irrelevance and redundancy of the features. Secondly, it offers the computational formulas for calculating interactive information. Finally, under the eleven data sets of UCI and three different classifiers, namely, KNN, SVM and C4.5, the DIFS algorithm increases the classification accuracy of the FullSet by 3.2848% and averagely decreases the number of features selected by 15.137. Hence, the DIFS algorithm can not only identify the relevance feature effectively, but also identify the irrelevant and redundant features. Moreover, it can effectively improve the classification accuracy of the data sets and reduce the feature dimensions of the data sets.

Evolutionary algorithms, including genetic algorithm, usually appear premature convergence. In this paper, new crossover and mutation operators are introduced. This paper addresses an application of improved genetic algorithms (IGA) for solving the Vehicle Routing Problem (VRP). After the introduction of new genetic operators, diversity of population becomes abundant. The ability of global search is obviously improved in new algorithm. Simulations indicate that new genetic algorithm is competitive with other modern heuristics.

Dynamic Adaboost learning with feature selection based on parallel genetic algorithm for image annotation

Feature selection is the process of choosing, or removing, features to obtain the most informative feature subset of minimal size. Such subsets are used to improve performance of machine learning algorithms and enable human understanding of the results. Approaches to feature selection in literature exploit several optimization algorithms. Multi-objective methods also have been proposed, minimizing at the same time the number of features and the error. While most approaches assess error resorting to the average of a stochastic K-fold cross-validation, comparing averages might be misleading. In this paper, we show how feature subsets with different average error might in fact be non-separable when compared using a statistical test. Following this idea, clusters of non-separable optimal feature subsets are identified. The performance in feature selection can thus be evaluated by verifying how many of these optimal feature subsets an algorithm is able to identify. We thus propose a multi-objective optimization approach to feature selection, EvoFS, with the objectives to i. minimize feature subset size, ii. minimize test error on a 10-fold cross-validation using a specific classifier, iii. maximize the analysis of variance value of the lowest-performing feature in the set. Experiments on classification datasets whose feature subsets can be exhaustively evaluated show that our approach is able to always find the best feature subsets. Further experiments on a high-dimensional classification dataset, that cannot be exhaustively analyzed, show that our approach is able to find more optimal feature subsets than state-of-the-art feature selection algorithms.

Hyperspectral images (HSIs) generally contain a large amount of spectral bands (features), and the redundant information in them will cause the Hughes phenomenon in the classification process. And feature extraction and feature selection are the two main existing methods to effectively reduce the redundancy of spectral information in the field of HSIs classification. Compared with feature extraction methods, feature selection methods can preserve most of the features of the original HSIs data without losing their valuable details. However, most existing feature selection methods based on single scene (domain) perform poorly in some scenes (domains) with insufficient labeled samples. Therefore, how to adopt an efficient feature selection method to select the optimal feature subsets of source scene and target scene and use the sample information of source scene to assist in the classification of target scene so as to improve the classification accuracy of images in the target scene as much as possible is still very challenging. In order to solve the above problem, this paper proposes a new cross-scene algorithm: Improved Ant Colony Optimization Algorithm-Based Cross-Scene Feature Selection Algorithm (IMACO-CSFS). In order to obtain more accurate feature subsets of the two scenes, IMACO-CSFS proposes a priority sorting-based ant colony strategy to make the subsequent search process focus on the global optimal solution (optimal feature subset) found in the previous iteration. In addition, in order to further accelerate the convergence speed of the global optimal solution, an ant colony strategy based on elite ants is proposed in IMACO-CSFS to more efficiently obtain the optimal feature subsets of the two scenes for training the classifier. Furthermore, this paper simultaneously considers overall classification accuracies of the optimal feature subsets for both scenes and dynamically adjusts their scale to ensure the consistency of the selected features between the two scenes, attenuating the effect of spectral shift and achieving the higher image classification accuracy in the target scene. Experimental results on three cross-scene HSI data pairs demonstrate that IMACO-CSFS is superior in cross-scene feature selection and cross-scene HSIs classification.

Proteins are cellular macromolecules made up of linear chains of amino acids that adopt a unique three-dimensional structure to carry out specific biological functions. Amongst various computational approaches developed for Protein Structure Prediction (PSP), ab initio methods perform the prediction without prior knowledge of known structures. However, due to huge computational complexity incurred by these methods, simplified hydrophobic-polar (HP) models are often used for the investigations. Despite this simplicity, PSP continues to be a hard combinatorial optimization problem requiring development of efficient heuristic search techniques. Memetic algorithm (MA), with its flexible architecture, encompasses the strength of both local and global search and has shown promise in solving problems with complex search landscape, including PSP. This thesis is focused on developing an effective ab initio technique under a suitable MA framework for solving the PSP problem using low resolution HP models. To enhance the effectiveness of the MA, a mechanism is devised to include knowledge from the problem domain by exploiting the concept of hydrophobic-core (H-core) formation. The size of the maximal possible H-core is estimated and the approximation is validated by our proposed deterministic search technique. Further, a new knowledge-based initial population generation technique ensures generating conformations which are not only valid, but also maintain diversity in the population. This helps to commence the search with good seeds leading towards faster convergence. While solving the PSP problem using MA, the greatest intricacy arises from the complex nature of its multi-modal landscape. Like all the evolutionary algorithms (EAs), MAs have the tendency to get trapped into local minima due to selection pressure and genetic drift induced by the genetic operators, and techniques are necessary to surmount this difficulty. A novel parental selection technique called “Adaptive Strategy for Assortative Mating” is developed that dynamically distributes the re production opportunity among the individuals of the population to boost the effectiveness of cross-over. Further, a new survival selection strategy, namely “Sib-based Survival Selection”, prevents the rapid spread of genetic material from a particular individual to other members in the population, ensuring the concurrent preservation of several potential solutions, and restricting premature convergence. A new diversification mechanism, namely “Memory-based Diversification”, systematically diversifies the population by introducing new genetic material whenever it gets stuck at local peaks and allows it to proceed for further progress. An extensive experimental analysis is also reported on the influence of all three techniques proposed to maintain the diversity of population. Furthermore, in order to carry out the conformational search efficiently, especially with the increase in sequence length, the optimization technique is designed such that it provides an effective balance between exploration and exploitation. A robust Multimodal Memetic Framework has been developed where optimization is accomplished in hierarchical stages. The population of each stage is partitioned into three different states: an Exploratory state to locate promising peaks, an Exploitative state to fine-tune the solutions found in the Exploratory state of the previous stage, and finally, a Central state to carry out the formation of potential solutions to explore the undiscovered regions. Non-identical genetic operators have been implemented in each state to balance the exploration and exploitation by employing necessary selection pressure and diversity. A non-isomorphic encoding based technique has been developed to capture the sub-structures that are representing a particular region. Furthermore, a novel local search technique exploits already explored fit individuals by employing the knowledge of sub-structures. The efficacy of the proposed framework is demonstrated with an adequate number of distinctive data sets and comparison with state-of-the-art methods.

Improved Salp Swarm Algorithm based on opposition based learning and novel local search algorithm for feature selection

Problem statement: Spam is an irrelevant or inappropriate message sent on the internet to a large number of newsgroups or users. A spam word is a list of well-known words that often appear in spam mails. Bloom Filter (BF) is used for identification of spam word. Approach: BF is a simple but powerful data structure that can check membership to a static set. The trade-off to use BF is a certain configurable risk of false positives. The odds of a false positive can be made very low if the hash bitmap is sufficiently large. Bin Bloom Filter (BBF) has number of BFs which assign group of words into bins with different false positive rates based on weight of the spam words. Genetic Algorithm (GA) was employed to minimize the total membership invalidation cost of BBF. GA had premature convergence problem. Simulated Annealing (SA) was incorporated with GA to prevent the premature convergence effectively. Results: The experimental results of total membership invalidation cost are analyzed for various sizes of bins. The results showed that the combined GA-SA model outperforms SA and GA model. Conclusion: GA has premature convergence due to its genetic operators that are not able to generate offsprings which are superior to the parents. So more number of similar chromosomes presented on the population. When GA is incorporated with SA new genes were introduced which causes diversity in the population and prevents premature convergence. The combined GA-SA outperforms GA and SA.

We review the recent work done by our group on theoretically analyzing the behavior of genetic algorithms(GAs) and searching for new ideas and strategies to design more efficient GAs. By introducing the concept of degree of population diversity and using the tools from finite Markov chain, we presented fairly clear characterizations of premature convergence in GAs and proved some results on the relationships between premature convergence and GA’s control parameters such as the population size, the mutation and crossover probabilities, and some population statistics. In the mean time, we have also obtained a result that depicts the search ability of the crossover operators in GAs. An upper bound on the convergence rate for the GAs has been established. Based on the theoretical analyses, we have proposed a novel GA called Genetic Algorithm with No Genetic Operators (GANGO), which needs neither to maintain a population nor to use the conventional genetic operators, and yet has the same search mechanisms as the classical GAs. As another development in improving the performance of GAs, a new concept called Dual Fitness Function was proposed which, when incorporated into the classical GAs will enable the algorithms to judge whether the current population is in the local or global attractive basins. And in this way, the control parameters of GAs can be adjusted adaptively. Some applications of our results to fields such as fuzzy systems, multi-destination routing problem (MDR) are briefly

Genetic algorithms have a problem in premature convergence, which causes the population of prospective solutions to the genetic algorithms to be trapped in suboptimum values. This premature convergence occurs due to a lack of speciation or diversity of the population of prospective solutions. To overcome premature convergence, increasing the diversity of the population of prospective solutions is necessary. Diversity makes genetic algorithms run smoothly to provide recommendations or personalize content in crowdsourcing media monitoring systems. This research attempts to overcome the premature convergence of genetic algorithms by adopting the CRISPR / Cas9 approach. CRISPR / Cas9 is a genome editing method that can edit targeted genes with template genes. The goal is to increase the diversity of the prospective solutions population, hoping that premature convergence does not occur. As a result, CRISPR / Cas9 has proven successful in overcoming premature convergence. However, more processing time and memory are needed than conventional genetic algorithms.

Microarray datasets often contain many features and incomplete values. To address these issues, this paper introduces a method called Genetic Algorithm-Based Adaptive Feature Selection with Missing value Imputation (GAFSMI) with two contributions. First, for identifying the noteworthy features, Genetic Algorithm-Based Adaptive Feature Selection (GAFS) is proposed. Then for imputing the non-ignorable missing values, Bayesian Genetic Algorithm (BAGEL) integrating genetic algorithm with Bayesian principles is introduced. These two pre-processing steps generate the complete dataset with optimal feature subset to perform classification with better accuracy. The proposed algorithm is implemented on eight microarray datasets and it is observed that GAFS selects optimal feature subset with appreciable classification accuracy than other feature selection techniques. The imputation accuracy of BAGEL measured is found to be better than other standard imputation techniques at different missing rates (5% to 40%). Classification accuracy is improved in all the datasets processed with GAFS and BAGEL.