Automatic Neural Architecture Search Based on an Estimation of Distribution Algorithm for Binary Classification of Image Databases

Univariate Marginal Distribution Algorithms (UMDAs) are a kind of Estimation of Distribution Algorithms (EDAs) which do not consider the dependencies among the variables. In this paper, on the basis of our proposed approach in [1], we present a rigorous proof for the result that the UMDA with margins (in [1] we merely showed the effectiveness of margins) cannot find the global optimum of the TRAPLEADINGONES problem [2] within polynomial number of generations with a probability that is super-polynomially close to 1. Such a theoretical result is significant in sheding light on the fundamental issues of what problem characteristics make an EDA hard/easy and when an EDA is expected to perform well/poorly for a given problem.

Estimation of distribution algorithms (EDAs) are intelligent systems that take advantage of statistical learning techniques. The distribution of promising regions in the search space is estimated and probabilistically guide the new particles’ searching towards them in EDAs. But EDAs cannot solve complex optimisation problems reliably and efficiently because of premature convergence and difficulty of complex probabilistic model learning. This paper presents a PCA-EDA algorithm which introduces principal component analysis (PCA) into simple Gaussian model-based EDA. PCA-EDA is aimed to keep the balance of accuracy and efficiency of probabilistic model learning, as well as to avoid premature convergence by PCA’s ability of analysis of Gaussian model’s variance. From the results of numerical experiments, it is showed that the proposed method is feasible and effective for complex optimisation problems.

Proper tuning of hyper-parameters is essential to the successful application of SVM-classifiers. Several methods have been used for this problem: grid search, random search, estimation of distribution Algorithms (EDAs), bio-inspired metaheuristics, among others. The objective of this paper is to determine the optimal method among those that recently reported good results: Bat algorithm, Firefly algorithm, Fruit-fly optimization algorithm, particle Swarm optimization, Univariate Marginal Distribution Algorithm (UMDA), and Boltzmann-UMDA. The criteria for optimality include measures of effectiveness, generalization, efficiency, and complexity. Experimental results on 15 medical diagnosis problems reveal that EDAs are the optimal strategy under such criteria. Finally, a novel performance index to guide the optimization process, that improves the generalization of the solutions while maintaining their effectiveness, is presented.

Estimation of distribution algorithms (EDAs) are a successful branch of evolutionary algorithms (EAs) that evolve a probabilistic model instead of a population. Analogous to genetic drift in EAs, EDAs also encounter the phenomenon that the random sampling in the model update can move the sampling frequencies to boundary values not justified by the fitness. This can result in a considerable performance loss. This work gives the first tight quantification of this effect for three EDAs and one ant colony optimizer, namely for the univariate marginal distribution algorithm, the compact genetic algorithm, population-based incremental learning, and the max-min ant system with iteration-best update. Our results allow to choose the parameters of these algorithms in such a way that within a desired runtime, no sampling frequency approaches the boundary values without a clear indication from the objective function. This paper for the Hot-off-the-Press track at GECCO 2020 summarizes the work Sharp Bounds for Genetic Drift in Estimation of Distribution Algorithms by B. Doerr and W. Zheng, which has been accepted for publication in the IEEE Transactions on Evolutionary Computation [5].

This paper investigates how the Univariate Marginal Distribution Algorithm (UMDA) behaves in non-stationary environments when engaging in sampling and selection strategies designed to correct diversity loss. Although their performance when solving Dynamic Optimization Problems (DOP) is less studied than population-based Evolutionary Algorithms, UMDA and other Estimation of Distribution Algorithms may follow similar schemes when tracking moving optima: genetic diversity maintenance, memory schemes, niching methods, and even reinicialization of the probability vectors. This study is focused on diversity maintenance schemes. A new update strategy for UMDA's probability model, based on Ant Colony Optimization transition probability equations, is presented and empirically compared with other strategies recently published that aim to correct diversity loss in UMDA. Results demonstrate that loss correction strategies delay or avoid full convergence, thus increasing UMDA's adaptability to changing environments. However, the strategy proposed in this paper achieves a higher performance on the DOP test set when compared with other methods. In addition, the new strategy incorporates two parameters that control the diversity of the probability model.

Estimation of distribution algorithm (EDA) is a new class of evolutionary algorithms with a wide range of real-world applications. However, it has been well known that the performance of EDA is not satisfactory enough if its population size is small. But to simply increase its population size may result in slow convergence. To the best knowledge of the authors', very few work has been done on improving the performance of EDA under small population size. This paper illustrates why EDA does not work well under small population size and proposes a novel approach termed as Over-Selection to boost EDA under small population size. Experimental results on several benchmark problems demonstrate that Over-Selection based EDA is often able to achieve a better solution without significantly increasing its time consumption when compared with the original version of EDA.

Evolutionary algorithms and elliptical copulas applied to continuous optimization problems

Feature subset selection for unsupervised learning, is a very important topic in artificial intelligence because it is the base for saving computational resources. In this implementation we use a typical testor's methodology in order to incorporate an importance index for each variable. This paper presents the general framework and the way two hybridized meta-heuristics work in this NP-complete problem. The evolutionary mechanisms are based on the Univariate Marginal Distribution Algorithm (UMDA) and the Genetic Algorithm (GA). GA and UMDA --- Estimation of Distribution Algorithm (EDA) use a very useful rapid operator implemented for finding typical testors on a very large dataset and also, both algorithms, have a local search mechanism for improving time and fitness. Experiments show that EDA is faster than GA because it has a better exploitation performance; nevertheless, GA' solutions are more consistent.

Existing neural architecture search (NAS) methods comprise linear connected convolution operations and use ample search space to search task-driven convolution neural networks (CNN). These CNN models are computationally expensive and diminish the quality of receptive fields for tasks like micro-expression recognition (MER) with limited training samples. Therefore, we propose a refined neural architecture search strategy to search for a tiny CNN architecture for MER. In addition, we introduced a refined hybrid module (RHM) for inner-level search space and an optimal path explore network (OPEN) for outer-level search space. The RHM focuses on discovering optimal cell structures by incorporating a multilateral hybrid spatiotemporal operation space. Also, spatiotemporal attention blocks are embedded to refine the aggregated cell features. The OPEN search space aims to trace an optimal path between the cells to generate a tiny spatiotemporal CNN architecture instead of covering all possible tracks. The aggregate mix of RHM and OPEN search space availed the NAS method to robustly search and design an effective and efficient framework for MER. Compared with contemporary works, experiments reveal that the RNAS-MER is capable of bridging the gap between NAS algorithms and MER tasks. Furthermore, RNAS-MER achieves new state-of-the-art performances on challenging MER benchmarks, including 0.8511%, 0.7620%, 0.9078% and 0.8235% UAR on COMPOSITE, SMIC, CASME-II and SAMM datasets respectively.

In this paper, we present an interference and channel conditions aware multiple relay assignment and subcarrier allocation scheme for OFDMA based cognitive radio systems employing cooperative transmission using Decode and Forward (DF) technique. We focus on the problem of assigning the relays, the relay powers and allocating the subcarriers to the destination nodes using the sum capacity of the cognitive radio system as the objective function to be maximized under the constraint of maximum acceptable levels of interference for the primary users (PU). The computational complexity of this assignment using exhaustive search algorithm grows exponentially with the number of relays, subcarriers and receivers. Thus we propose Estimation of Distribution Algorithm (EDA) for this assignment problem. EDA is a probabilistic evolutionary algorithm which updates its population at each iteration on the basis of the probability densities obtained from the population of superior candidates evaluated and chosen at the previous iteration. EDA employed with our allocation scheme has low computational complexity and its performance is comparable to the Upper Bound of the solution as shown by various simulation results.

A newly proposed distributed dynamic state estimation algorithm based on the maximum a posteriori (MAP) technique is generalised and studied for power systems. The system model involves linear time-varying load dynamics and nonlinear measurements. The main contribution of this paper is to compare the performance and feasibility of this distributed algorithm with several existing distributed state estimation algorithms in the literature. Simulations are tested on the IEEE 39-bus and 118-bus systems under various operating conditions. The results show that this distributed algorithm performs better than distributed quasi-steady state estimation algorithms which do not use the load dynamic model. The results also show that the performance of this distributed method is very close to that by the centralized state estimation method. The merits of this algorithm over the centralized method lie in its low computational complexity and low communication load. Hence, the analysis supports the efficiency and benefits of the distributed algorithm in applications to large-scale power systems.

Finding a good model and efficiently estimating the distribution is still an open challenge in estimation of distribution algorithms (EDAs). Factorization encoded by models in most of the EDAs are constrained. However for optimization of many real-world problems, finding the model capable of representing complex interactions without much computational complexity overhead is the key challenge. On the other hand factor graph which is the most natural graphical model for representing additively decomposable functions is rarely employed in EDAs. In this paper we introduce Factor Graph based EDA (FGEDA) which learns factor graph as the model and estimate the probability distribution represented by the learned factor graph using Markov blanket canonical factorization. The class of factorization that is employed for approximation of distribution in FGEDA is expanded relative to famous EDAs. We have used matrix factorization for learning the factor graph of the problem based on the pairwise mutual information between pair of variables. Gibbs sampling and BB-wise crossover are used to generate new samples. Empirical evaluation as well as theoretical analysis of the approach show the efficiency and power of FGEDA in the optimization of functions with complex interactions. It is showed experimentally that FGEDA outperform other well-known EDAs.

The Pareto front (Pareto set) of a continuous optimization problem with m objectives is a (m-1) dimensional piecewise continuous manifold in the objective space (the decision space) under some mild conditions. Based on this regularity property in the objective space, we have recently developed several multiobjective estimation of distribution algorithms (EDAs). However, this property has not been utilized in the decision space. Using the regularity property in both the objective and decision space, this paper proposes a simple EDA for multiobjective optimization. Since the location information has not efficiently used in EDAs, a combination of EDA and differential evolution (DE) is suggested for improving the algorithmic performance. The hybrid method and the pure EDA method proposed in this paper, and a DE based method are compared on several test instances. Experimental results have shown that the algorithm with the proposed strategy is very promising.

A new class of estimation of distribution algorithms (EDAs), known as cellular EDAs (cEDAs), has recently emerged. In these algorithms, the population is decentralized by partitioning it into many small collaborating subpopulations, arranged in a toroidal grid, and interacting only with its neighboring subpopulations. In this work, we study the simplest cEDA —the cellular univariate marginal distribution algorithm (cUMDA). In an attempt to explain its behaviour, we extend the well known takeover time analysis usually applied to other evolutionary algorithms to the field of EDAs. We also give in this work empirical arguments in favor of using the cUMDAs instead of its centralized equivalent.

Identification of crops is an important topic in the agricultural domain. Hyperspectral remote sensing data are very useful for crop feature extraction and classification. Remote sensing data is an unstructured data and Convolutional Neural Network (CNN) can work on unstructured data efficiently. This paper presents an evaluation of CNN for crop classification using the Indian Pines standard dataset obtained from the AVIRIS sensor and the study area dataset obtained from the EO-1hyperion sensor. Optimized CNN has been tuned by training the model on different parameters. It has been compared with two classification algorithms: Deep Neural Network (DNN) and Convolutional Autoencoder. According to the test results, the proposed optimized CNN model provided better results as compared to the other two methods. CNN has given 97 ± 0.58% overall accuracy for the Indian Pines standard dataset and 78 ± 2.43% for our study area dataset.