A Maximum Entropy Approach to Recovering Information from Multinomial Response Data

This article introduces two new maximum entropy (ME) methods for modeling the distribution of time to an event. One method is within the classical ME framework and provides characterizations of change point models such as the piecewise exponential distribution. The second method uses the entropy of the equilibrium distribution (ED) for the objective function and provides new characterizations of the exponential, Weibull, Pareto, and uniform distributions. With the same moment constraints, the classical ME and the maximum ED entropy algorithms generate different models for the interarrival time. © 2014 Wiley Periodicals, Inc. Naval Research Logistics 61: 427–434, 2014

One well-recognized problem in three-dimensional (3D) confocal microscopic images is that the intensities in deeper slices are generally weaker than those in shallower slices. The loss of intensity with depth hampers both qualitative observation and quantitative measurement of specimens. Two major types of methods exist to compensate for this intensity loss: the first is based on the geometrical optics inside the specimen, and the second applies an empirical parametric intensity decay function (IDF) of depth. A common feature shared by both methods is that they are parameter-dependent. However, for the optics-based method there are as yet no fully automated parameter-setting approaches; and for the IDF method the traditional profile-fitting approach cannot provide proper parameters if the presumed IDF model does not match the experimental intensity-depth profile of the 3D image. In this paper, we propose a novel maximum-entropy (ME) approach to fully automated parameter-setting. In principle the ME approach is suitable for any compensation method as long as it is parameter-dependent. The basic assumption is that without intensity loss an ideal 3D image should be generally homogeneous with respect to depth and this axial homogeneity can be represented by the entropy of a normalized intensity-depth profile. Experiments on real confocal images showed that such a profile was consistent with visual evaluation of axial intensity homogeneity and that the ME approach could provide proper parameters for both compensation methods mentioned above. Moreover, for the IDF method, experiments on both real and simulated data showed that the ME approach could provide more precise parameters than with traditional profile-fitting. The Appendix provides a proof that under certain conditions the global maximization of the profile-entropy is guaranteed.

Reconstruction of calmodulin single-molecule FRET states, dye interactions, and CaMKII peptide binding by MultiNest and classic maximum entropy

Predicting multinomial choices using maximum entropy

During the MaxEnt 2002 workshop in Moscow, Idaho, Tony Vignaux asked again a few simple questions about using Maximum Entropy or Bayesian approaches for the famous Dice problems which have been analyzed many times through this workshop and also in other places. Here, there is another analysis of these problems. I hope that, this paper will answer a few questions of Tony and other participants of the workshop on the situations where we can use Maximum Entropy or Bayesian approaches or even the cases where we can actually use both of them.

Intrusion detection is a critical component of secure information systems. Current intrusion detection systems (IDS) especially NIDS (Network Intrusion Detection System) examine all data features to detect intrusions. However, some of the features may be redundant or contribute little to the detection process and therefore they have great impact on the system performance. This paper proposes a lightweight intrusion detection model that is computationally efficient and effective based on feature selection and Maximum Entropy (ME) model. Firstly, the issue of identifying important input features is addressed. Since elimination of the insignificant and/or useless inputs leads to a simplification of the problem, therefore results to faster and more accurate detection. Secondly, classic ME model is used to learn and detect intrusions using the selected important features. Experimental results on the well-known KDD 1999 dataset show the proposed model is effective and can be applied to real-time intrusion detection environments.

Price responsiveness of herbicide demand in corn for farmers in Indiana's White River Basin using cross-section data from individual farms is estimated. Particular attention is paid to appropriate treatment of binding nonnegativity constraints. Estimation was first attempted using an approach to demand systems estimation suggested by Lee and Pitt. However, analytical and computational difficulties effectively preclude estimation by the Lee and Pitt approach. As an alternative, a maximum entropy (ME) approach is presented and discussed. Results from the ME estimator tentatively indicate limited response of herbicide demand to changes in own prices. The maximum entropy approach to demand systems estimation appears to have merit and warrants further attention.

The objective of this paper is to evaluate the degree of financial integration achieved in the European Union based on covered interest parity and using Generalized Maximum Entropy. EU countries are divided into two groups according to their current situation with respect to the adoption of the euro. Financial integration before the adoption of the euro is analyzed for the countries that adopted the euro in 1999. Similarly, current financial integration is evaluated for non-euro EU countries. Besides the importance of comparing the situation of the non-euro EU countries with the situation of the euro EU countries previous to the euro adoption, which may be useful to evaluate an eventual decision of the non-euro members to adopt the euro, it is interesting to analyze the performance of Generalized Maximum Entropy. Generalized Maximum Entropy has the ability to estimate the parameters of a regression model without imposing any constrains on the probability distribution of errors and it is robust even when we have ill-posed problems. Overall our results suggest that the degree of financial integration on non-euro countries is lower than the degree of financial integration that existed among euro adopting countries before the adoption of the euro.

Biases in Willingness-to-Pay Measures from Multinomial Logit Estimates Due to Unobserved Heterogeneity

A generalized maxentropic inversion procedure for noisy data

Multicollinearity hampers empirical econometrics. The remedies proposed to date suffer from pitfalls of their own. The ridge estimator is not generally accepted as a vital alternative to the ordinary least-squares (OLS) estimator because it depends upon unknown parameters. The generalized maximum entropy (GME) estimator of Golan, Judge and Miller depends upon subjective exogenous information that affects the estimated parameters in an unpredictable way. This paper presents novel maximum entropy estimators inspired by the theory of light that do not depend upon any additional information. Monte Carlo experiments show that they are not affected by any level of multicollinearity and dominate OLS uniformly. The Leuven estimators are consistent and asymptotically normal.

Location‐only and use‐availability data: analysis methods converge

In this article, we provide initial findings regarding the problem of solving likelihood equations by means of a maximum entropy (ME) approach. Unlike standard procedures that require equating the score function of the maximum likelihood problem at zero, we propose an alternative strategy where the score is instead used as an external informative constraint to the maximization of the convex Shannon’s entropy function. The problem involves the reparameterization of the score parameters as expected values of discrete probability distributions where probabilities need to be estimated. This leads to a simpler situation where parameters are searched in smaller (hyper) simplex space. We assessed our proposal by means of empirical case studies and a simulation study, the latter involving the most critical case of logistic regression under data separation. The results suggested that the maximum entropy reformulation of the score problem solves the likelihood equation problem. Similarly, when maximum likelihood estimation is difficult, as is the case of logistic regression under separation, the maximum entropy proposal achieved results (numerically) comparable to those obtained by the Firth’s bias-corrected approach. Overall, these first findings reveal that a maximum entropy solution can be considered as an alternative technique to solve the likelihood equation.

A maximum entropy approach to adaptive statistical language modelling

Remote sensing using passive sonar in the ocean is a challenging problem due to variations in the geoacoustic structure of the seabed and unknown source location and strength. One way to improve remote sensing is to perform an optimization for geoacoustic and source parameters. We use a Bayesian maximum entropy (BME) approach with a viscous-grain shearing parameterization for two sediment layers. The statistical optimization provides probability distributions for porosity and thickness of the sediment layers as well as ship speed, closest point of approach, and the source strength for the Wales-Heitmeyer empirical source level spectrum. We use this approach on spectrograms of transiting ships collected on a vertical line array during the 2017 seabed characterization experiment. We compare the resulting parameter distributions from distinct ships as well as previous estimates of geoacoustic values and source properties. This research shows that the BME approach obtains estimates for porosity and source strength that have narrow posterior probability distributions. [Work supported by the Office of Naval Research.]