Adaptive estimated maximum-entropy distribution model

Estimation of distribution algorithms (EDAs) are intelligent systems that take advantage of statistical learning techniques. The distribution of promising regions in the search space is estimated and probabilistically guide the new particles’ searching towards them in EDAs. But EDAs cannot solve complex optimisation problems reliably and efficiently because of premature convergence and difficulty of complex probabilistic model learning. This paper presents a PCA-EDA algorithm which introduces principal component analysis (PCA) into simple Gaussian model-based EDA. PCA-EDA is aimed to keep the balance of accuracy and efficiency of probabilistic model learning, as well as to avoid premature convergence by PCA’s ability of analysis of Gaussian model’s variance. From the results of numerical experiments, it is showed that the proposed method is feasible and effective for complex optimisation problems.

Location‐only and use‐availability data: analysis methods converge

Estimation of Distribution Algorithms (EDAs) is new kinds of colony evolution algorithms. It produces its new generation by constructing probability distribution model through counting excellent information of individuals of present colony EDAs first, and then sampling the model. To solve the NP-Hard question as EDAs searching optimum network structure, a new Maximum Entropy Estimation of Distribution Algorithm (MEEDA) is proposed; and the paper puts forward the MEEDA to optimize two degree of freedom PID controller parameters. The simulation results show that the optimal 2-DOF PID controller has simultaneously both the characteristics of command tracking and disturbance rejection, so we can see the simulation verifies the effectiveness of the EDA algorithm.

This essay is, primarily, a discussion of four results about the principle of maximizing entropy (MAXENT) and its connections with Bayesian theory. Result1provides a restricted equivalence between the two: where the Bayesian model for MAXENT inference uses an “a priori“ probability that is uniform, and where all MAXENT constraints are limited to 0–1 expectations for simple indicator-variables. The other three results report on an inability to extend the equivalence beyond these specialized constraints. Result2established a sensitivity of MAXENT inference to the choice of the algebra of possibilities even though all empirical constraints imposed on the MAXENT solution are satisfied in each measure space considered. The resulting MAXENT distribution is not invariant over the choice of measure space. Thus, old and familiar problems with the Laplacian principle of Insufficient Reason also plague MAXENT theory. Result3builds upon the findings of Friedman and Shimony (1971; 1973) and demonstrates the absence of an exchangeable, Bayesian model for predictive MAXENT distributions when the MAXENT constraints are interpreted according to Jaynes's (1978) prescription for his (1963) Brandeis Dice problem. Lastly, Result4generalizes the Friedman and Shimony objection to cross-entropy (Kullback-information) shifts subject to a constraint of a new odds-ratio for two disjoint events.

The Estimation of Distribution Algorithms (EDAs) is a novel class of evolutionary algorithms which is motivated by the idea of building probabilistic graphical model of promising solutions to represent linkage information between variables in chromosome. Through learning of and sampling from probabilistic graphical model, new population is generated and optimization procedure is repeated until the stopping criteria are met. In this paper, the mechanism of the Estimation of Distribution Algorithms is analyzed. Currently existing EDAs are surveyed and categorized according to the probabilistic model they used, then the strengths and weakness and the future perspective of EDAs are concluded.

Markov random field (MRF) modelling techniques have been recently proposed as a novel approach to probabilistic modelling for estimation of distribution algorithms (EDAs). An EDA using this technique was called distribution estimation using Markov random fields (DEUM). DEUM was later extended to DEUMd. DEUM and DEUMd use a univariate model of probability distribution, and have been shown to perform better than other univariate EDAs for a range of optimization problems. This paper extends DEUM to use a bivariate model and applies it to the Ising spin glass problems. We propose two variants of DEUM that use different sampling techniques. Our experimental result show a noticeable gain in performance.

Multivariate maximum entropy identification, transformation, and dependence

How Important is the Detection of Changes in Active Constraints in Real-Time Optimization?

This chapter provides a survey of current applications of X-ray diffraction techniques in crystal structure analysis, with some focus on recent advances that have been made in the scope and potential for carrying out crystal structure determination directly from diffraction data. The basic concepts of crystal structure analysis, Rietveld refinement, and the concepts used for the estimation and analysis of charge density in a crystal are discussed. The more reliable models for charge density estimation like multipole formalism and maximum entropy method are discussed in detail. The local structural analysis technique atomic pair distribution function is also discussed.

In this paper, a new approach for motion generation and optimization of the flexible macro-micro manipulator system is proposed based on Estimation of Distribution Algorithm (EDA). The macro-micro manipulator system is a redundant system, of which inverse kinematics remains challenging, with no generic solution to date. Here, the manipulator system configurations, or the optimal joint motions, are generated using the EDA algorithm base on Gaussian probability model. Compared with simple genetic algorithms (SGA), this approach uses fewer parameters and the time for motion optimization is remarkably reduced. The proposed approach shows excellent performance on motion generation and optimization of a flexible macro-micro manipulator system, as demonstrated by the simulation results.

After a brief expository account of the Shannon-Jaynes principle of maximum entropy (POME) for discrete and continuous variables, we give here an account of some recent research work which (i) a “histogram” method to contrast the discrete and continuous modes computation and the role of histogram in actual practice when dealing with continuous probability distributions. (ii) The idea of mean logarithmic decrement associated with a probability distribution is introduced and is shown to be related to the concept of differential entropy. The mean with respect to an arbitrary probability distribution of the logarithm of the ratio of the probability density function for an exponential distribution is discussed in the context of hydrological investigations. Unlike the entropy of a continuous probability distribution introduced in (i), this quantity which is an example of the Kullback-Leibler (KL) Information, is always positive and invariant under coordinate transformation. (iii) The constraints entering into POME as well as a minimum K L information are identified as a class of sufficient statistics which determine the unknown parameters in the probability density functions that occur in the most commonly used hydrological models. (iv) An example of (iii) where only the first two moments in a semi-infinite domain are givens is discussed to shed light on the limitations of POME, recentky recognized by Wragg and coworkers, and is made relevant to the work of Sonuga on rainfall-run off relationship. Finally, (v) a method of generating probability distributions starting from one basic distribution employing coordinate transformations is given. This in conjuction with (iii) leads to the notions of “Physical constraints” in contrast to the “mathematical constraints” in examing parameter estimation.

Object-oriented resource-based planning method (ORPM) for construction

This paper concerns the busy period of a single server queueing model with exponentially distributed repeated attempts. Several authors have analyzed the structure of the busy period in terms of the Laplace transform but, the information about the density function is limited to first and second order moments. We use the maximum entropy principle to find the least biased density function subject to several mean value constraints. We perform results for three different service time distributions: 3-stage Erlang, hyperexponential and exponential. Also a numerical comparative analysis between the exact Laplace transform and the corresponding maximum entropy density is presented.

The approximate analytical method using the maximum entropy method (MEM) is proposed to estimate the stationary probability density function of nonlinear systems subjected to non-white random excitation. The MEM is often used to estimate the response distribution of nonlinear systems subjected to Gaussian white noise. In order to obtain the response distribution, the moment equations are used as the constraint conditions on the MEM. However, in the case of nonlinear systems under non-white random excitation, the term of the cross-correlation between the excitation and the response in the moment equations is generally not able to be calculated. In the proposed method, the equivalent linearization technique is applied to calculate the cross-correlation approximately. Using the method, we estimate the stationary response probability density functions of a Duffing oscillator and an asymmetric nonlinear system subjected to non-white excitation with the exponentially decaying correlation function. In the analysis, a wide range of the excitation bandwidth is considered. To demonstrate the effectiveness of the method, the results are compared with those of Monte Carlo simulation.

As a typical model-based evolutionary algorithm, estimation of distribution algorithm (EDA) possesses unique characteristics and has been widely applied in global optimization. However, the commonly used Gaussian EDA (GEDA) usually suffers from premature convergence, which severely limits its search efficiency. This paper first systematically analyzes the reasons for the deficiency of traditional GEDA, then tries to enhance its performance by exploiting the evolution direction, and finally develops a new GEDA variant named EDA2. Instead of only utilizing some good solutions produced in the current generation to estimate the Gaussian model, EDA2 preserves a certain number of high-quality solutions generated in the previous generations into an archive and employs these historical solutions to assist estimating the covariance matrix of Gaussian model. By this means, the evolution direction information hidden in the archive is naturally integrated into the estimated model, which in turn can guide EDA2 toward more promising solution regions. Moreover, the new estimation method significantly reduces the population size of EDA2 since it needs fewer individuals in the current population for model estimation. As a result, a fast convergence can be achieved. To verify the efficiency of EDA2, we tested it on a variety of benchmark functions and compared it with several state-of-the-art EAs. The experimental results demonstrate that EDA2 is efficient and competitive.