A neural net algorithm for multidimensional maximum entropy spectrum estimation

Maximum Entropy Spectral Analysis (MESA) Method is a powerful alternative to the traditionally used Blackman Tukey (BT) and Fast Fourier Transform (FFT) Methods in hydrologic and climatologic time series analysis. Maximum entropy spectra of annual rainfall time series measured at Mangalore and Waltair and monthly flow time series of Godavari and Krishna Rivers in India are presented and compared with the spectra obtained by using BT and FFT methods. A recently developed Bayesian decision rule is applied to determine the length of filters in MESA. Sensitivity of the maximum entropy spectra to the length of filter used is also discussed.

This paper describes efficient algorithms for determining how buffer space should be allocated in a flow line. We analyze two problems: a primal problem, which minimizes total buffer space subject to a production rate constraint; and a dual problem, which maximizes production rate subject to a total buffer space constraint. The dual problem is solved by means of a gradient method, and the primal problem is solved using the dual solution. Numerical results are presented. Profit optimization problems are natural generalizations of the primal and dual problems, and we show how they can be solved using essentially the same algorithms.

3 - Further Topics in Linear Programming

A number of modern spectral estimators are shown to have a common generic formulation. These include minimum variance, MUSIC, and maximum entropy. A new maximum entropy spectral estimator is derived using constraints on the modal powers or the expected-square projections of the data onto the eigenvectors of the data covariance matrix. The formulation incorporates uncertainty in the modal power constraints and the signal-versus-noise subspace separation. The resulting estimators have forms which incorporate all other modern estimators, including maximum entropy and minimum norm. The new estimators allow further development when a priori information is used in the constraints. Comparison of one version of the estimator with the minimum norm verifies the greater probability of resolution of the minimum norm but indicates in some instances the value of the incorporated uncertainties. Another version uses complex constraints and reduces to conventional maximum entropy or minimum norm under certain conditions.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

In this paper, we consider the Lyapunov-type linear programming and its dual over symmetric cones. By introducing and characterizing the generalized inverse of Lyapunov operator in Euclidean Jordan algebras, we establish two kinds of Lyapunov-type Farkas’ lemmas to exhibit feasibilities of the corresponding primal and dual programming problems, respectively. As one of the main results, we show that the feasibilities of the primal and dual problems lead to the solvability of the primal problem and zero duality gap under some mild condition. In this case, we obtain that any solution to the pair of primal and dual problems is equivalent to the solution of the corresponding KKT system.

In this paper we study the problem of minimizing total weighted tardiness, a proxy for maximizing on-time delivery performance, on parallel nonidentical batch processing machines. We first formulate the (primal) problem as a nonlinear integer programming model. We then show that the primal problem can be solved exactly by solving a corresponding dual problem with a nonlinear relaxation. Since both the primal and the dual problems are NP-hard, we use genetic algorithms, based on random keys and multiple choice encodings, to heuristically solve them. We find that the genetic algorithms consistently outperform a standard mathematical programming package in terms of solution quality and computation time. We also compare the smaller problem instances to a breadth-first tree search algorithm that gives evidence of the quality of the solutions.

: This tutorial paper describes the maximum entropy spectrum and the Burg technique for computing the prediction error power and prediction error filter coefficients in the associated spectral estimation formula. The maximum entropy spectrum is identical to the autoregressive spectral estimator. Also included in this paper is a discussion of the K-line spectrum, which is the wavenumber analogue of the frequency-domain maximum entropy spectrum, and the Burg technique modifications necessary for its implementation. The purpose of this paper is to provide a complete and self-contained account of the main features of the maximum entropy spectrum. Since many of the relevant mathematical derivations are not found in the formal published literature, they are incorporated in this paper. Supporting material and various sidelights of the maximum entropy spectrum appear in the appendices.

The probabilistic Description Logic Open image in new window extends the classical Description Logic Open image in new window with probabilistic conditionals of the form (D|C)[p] stating that “D follows from C with probability p.” Conditionals are interpreted based on the aggregating semantics where probabilities are understood as degrees of belief. For reasoning with probabilistic conditional knowledge bases in Open image in new window , the principle of maximum entropy provides a valuable methodology following the idea of completing missing information in a most cautious way. In this paper, we give a guidance on calculating approximations of the maximum entropy distribution for Open image in new window -knowledge bases. For this, we discuss the benefits of solving the dual maximum entropy optimization problem instead of the primal problem. In particular, we show how representations of approximations of the maximum entropy distribution can be derived from the dual problem in time polynomial in the size of the underlying domain. The domain is the crucial quantity in practical applications. For a compact representation of the objective function of the dual maximum entropy optimization problem, we apply the principle of typed model counting.

Abstract. In this paper, some omissions in Mishra and Lai [13], havebeen pointed out and their corrective measures have been discussed briefly. 1. IntroductionA pair of primal and dual problems in mathematical programming is calledsymmetric if the dual of the dual is the primal problem. Dorn [6] introducedthe concept of symmetric duality in quadratic programming. His results wereextended to nonlinear convex programming problems by Dantzig et al. [4] andlater by Bazaraa and Goode [3] over arbitrary cones.Mangasarian [11] introduced the concept of second-order duality for non-linear problems. Since then, many authors [1, 2, 7, 9, 15, 16] have workedon second-order symmetric duality. Mishra and Lai [13] studied Mond-Weirtype second-order multiobjective symmetric duality for the following pair ofproblems:(P) K−minimize f(x,y)−12p T ∇ yy f(x,y)psubject to −∇ y (λ T f)(x,y) −∇ yy (λ T f)(x,y) ∈ C ∗2 ,y T [∇ y (λ T f)(x,y) +∇ yy (λ T f)(x,y)] > 0,λ ∈ K ∗ , x ∈ C 1 .(D) K−maximize f(u,v)−12q

Nonlinear optimization problems have two different representations, the primal problem and the dual problem. The relation between the primal and the dual problem is provided by an elegant duality theory. This chapter presents the basics of duality theory. Section 4.1 discusses the primal problem and the perturbation function. Section 4.2 presents the dual problem. Section 4.3 discusses the weak and strong duality theorems, while section 4.4 discusses the duality gap. This section presents the formulation of the primal problem, the definition and properties of the perturbation function, the definition of stable primal problem, and the existence conditions of optimal multiplier vectors.

This chapter states the Monge–Kantorovich problem and provides the duality result in a fairly general setting. The primal problem is interpreted as the central planner's problem of determining the optimal assignment of workers to firms, while the dual problem is interpreted as the invisible hand's problem of obtaining a system of decentralized equilibrium prices. In general, the primal problem always has a solution (which means that an optimal assignment of workers to jobs exists), but the dual does not: the optimal assignment cannot always be decentralized by a system of prices. However, the cases where the dual problem does not have a solution are rather pathological, and in all of the cases considered in the rest of the book, both the primal and the dual problems have solutions.

In this paper, we consider a least square semidefinite programming problem under ellipsoidal data uncertainty. We show that the robustification of this uncertain problem can be reformulated as a semidefinite linear programming problem with an additional second-order cone constraint. We then provide an explicit quantitative sensitivity analysis on how the solution under the robustification depends on the size/shape of the ellipsoidal data uncertainty set. Next, we prove that, under suitable constraint qualifications, the reformulation has zero duality gap with its dual problem, even when the primal problem itself is infeasible. The dual problem is equivalent to minimizing a smooth objective function over the Cartesian product of second-order cones and the Euclidean space, which is easy to project onto. Thus, we propose a simple variant of the spectral projected gradient method (Birgin et al. in SIAM J. Optim. 10:1196---1211, 2000) to solve the dual problem. While it is well-known that any accumulation point of the sequence generated from the algorithm is a dual optimal solution, we show in addition that the dual objective value along the sequence generated converges to a finite value if and only if the primal problem is feasible, again under suitable constraint qualifications. This latter fact leads to a simple certificate for primal infeasibility in situations when the primal feasible set lies in a known compact set. As an application, we consider robust correlation stress testing where data uncertainty arises due to untimely recording of portfolio holdings. In our computational experiments on this particular application, our algorithm performs reasonably well on medium-sized problems for real data when finding the optimal solution (if exists) or identifying primal infeasibility, and usually outperforms the standard interior-point solver SDPT3 in terms of CPU time.

Stochastic linear–quadratic control problems with affine constraints

Duality is an essential part of LP theory. It is related to the special relationship between one LP problem and another, both of which involve the same data (A, b, c), but positioned differently (except for the self-duality, see below). The former is referred to as primal problem, and the latter as dual problem. Based on duality, once a primal problem is solved, so is the dual problem, and vice versa. This chapter is devoted to such topics as the relationship between their feasible regions as well as optimal solutions, optimality condition, dual simplex algorithm in tableau form and in revised form, shadow price, and dual elimination. Before making notes on the duality in the final section, we introduce a special bilevel LP problem together with a related approach to intercepting the optimal set.

We present an extension to Jaynes’ maximum entropy principle that incorporates latent variables. The principle oflatent maximum entropywe propose is different from both Jaynes’ maximum entropy principle and maximum likelihood estimation, but can yield better estimates in the presence of hidden variables and limited training data. We first show that solving for a latent maximum entropy model poses a hard nonlinear constrained optimization problem in general. However, we then show that feasible solutions to this problem can be obtained efficiently for the special case of log-linear models---which forms the basis for an efficient approximation to the latent maximum entropy principle. We derive an algorithm that combines expectation-maximization with iterative scaling to produce feasible log-linear solutions. This algorithm can be interpreted as an alternating minimization algorithm in the information divergence, and reveals an intimate connection between the latent maximum entropy and maximum likelihood principles. To select a final model, we generate a series of feasible candidates, calculate the entropy of each, and choose the model that attains the highest entropy. Our experimental results show that estimation based on the latent maximum entropy principle generally gives better results than maximum likelihood when estimating latent variable models on small observed data samples.