In combinatorial solution spaces Iterated Local Search (ILS) turns out to be exceptionally successful. The question arises: is ILS also capable of improving the optimization process in continuous solution spaces? To demonstrate that hybridization leads to powerful techniques in continuous domains, we introduce a hybrid meta-heuristic that integrates Powell’s direct search method. It combines direct search with elements from population based evolutionary optimization. The approach is analyzed experimentally on a set of well known test problems and compared to a state-of-the-art technique, i.e., a restart variant of the Covariance Matrix Adaptation Evolution Strategy with increasing population sizes (G-CMA-ES). It turns out that the population-based Powell-ILS is competitive to the CMA-ES, in some cases even significantly faster and behaves more robust than the pure strategy of Powell in multimodal fitness landscapes. Further experiments on the perturbation mechanism, population sizes, and problems with noise complete the analysis of the hybrid methodology and lead to parameter recommendations.

Learning strategies under covariate shift have recently been widely discussed. The density of learning inputs under covariate shift is different from that of test inputs. Learning machines in such environments need to employ special learning strategies to acquire greater capabilities of generalizing through learning. However, incremental learning methods are also used for learning in non-stationary learning environments, which represent a kind of covariate shift. However, the relation between covariate-shift environments and incremental-learning environments has not been adequately discussed. This paper focuses on the covariate shift in incremental-learning environments and our re-construction of a suitable incremental-learning method. Then, the model-selection criterion is also derived, which is to be an essential object function for memetic algorithms to solve these kinds of learning problems.

We propose two methods for tuning membership functions of a kernel fuzzy classifier based on the idea of SVM (support vector machine) training. We assume that in a kernel fuzzy classifier a fuzzy rule is defined for each class in the feature space. In the first method, we tune the slopes of the membership functions at the same time so that the margin between classes is maximized under the constraints that the degree of membership to which a data sample belongs is the maximum among all the classes. This method is similar to a linear all-at-once SVM. We call this AAO tuning. In the second method, we tune the membership function of a class one at a time. Namely, for a class the slope of the associated membership function is tuned so that the margin between the class and the remaining classes is maximized under the constraints that the degrees of membership for the data belonging to the class are large and those for the remaining data are small. This method is similar to a linear one-against-all SVM. This is called OAA tuning. According to the computer experiment for fuzzy classifiers based on kernel discriminant analysis and those with ellipsoidal regions, usually both methods improve classification performance by tuning membership functions and classification performance by AAO tuning is slightly better than that by OAA tuning.

In this paper, we identify a number of topics relevant for the improvement and development of discrete estimation of distribution algorithms. Focusing on the role of probability distributions and factorizations in estimation of distribution algorithms, we present a survey of current challenges where further research must provide answers that extend the potential and applicability of these algorithms. In each case we state the research topic and elaborate on the reasons that make it relevant for estimation of distribution algorithms. In some cases current work or possible alternatives for the solution of the problem are discussed.