Yet Another Analysis of Dice Problems

One well-recognized problem in three-dimensional (3D) confocal microscopic images is that the intensities in deeper slices are generally weaker than those in shallower slices. The loss of intensity with depth hampers both qualitative observation and quantitative measurement of specimens. Two major types of methods exist to compensate for this intensity loss: the first is based on the geometrical optics inside the specimen, and the second applies an empirical parametric intensity decay function (IDF) of depth. A common feature shared by both methods is that they are parameter-dependent. However, for the optics-based method there are as yet no fully automated parameter-setting approaches; and for the IDF method the traditional profile-fitting approach cannot provide proper parameters if the presumed IDF model does not match the experimental intensity-depth profile of the 3D image. In this paper, we propose a novel maximum-entropy (ME) approach to fully automated parameter-setting. In principle the ME approach is suitable for any compensation method as long as it is parameter-dependent. The basic assumption is that without intensity loss an ideal 3D image should be generally homogeneous with respect to depth and this axial homogeneity can be represented by the entropy of a normalized intensity-depth profile. Experiments on real confocal images showed that such a profile was consistent with visual evaluation of axial intensity homogeneity and that the ME approach could provide proper parameters for both compensation methods mentioned above. Moreover, for the IDF method, experiments on both real and simulated data showed that the ME approach could provide more precise parameters than with traditional profile-fitting. The Appendix provides a proof that under certain conditions the global maximization of the profile-entropy is guaranteed.

The classical maximum entropy (ME) approach to estimating the unknown parameters of a multinomial discrete choice problem, which is equivalent to the maximum likelihood multinomial logit (ML) estimator, is generalized. The generalized maximum entropy (GME) model includes noise terms in the multinomial information constraints. Each noise term is modeled as the mean of a finite set of a priori known points in the interval [−1,1] with unknown probabilities where no parametric assumptions about the error distribution are made. A GME model for the multinomial probabilities and for the distributions associated with the noise terms is derived by maximizing the joint entropy of multinomial and noise distributions, under the assumption of independence. The GME formulation reduces to the ME in the limit as the sample grows large or when no noise is included in the entropy maximization. Further, even though the GME and the logit estimators are conceptually different, the dual GME model is related to a gener...

Price responsiveness of herbicide demand in corn for farmers in Indiana's White River Basin using cross-section data from individual farms is estimated. Particular attention is paid to appropriate treatment of binding nonnegativity constraints. Estimation was first attempted using an approach to demand systems estimation suggested by Lee and Pitt. However, analytical and computational difficulties effectively preclude estimation by the Lee and Pitt approach. As an alternative, a maximum entropy (ME) approach is presented and discussed. Results from the ME estimator tentatively indicate limited response of herbicide demand to changes in own prices. The maximum entropy approach to demand systems estimation appears to have merit and warrants further attention.

Location‐only and use‐availability data: analysis methods converge

In this article, we provide initial findings regarding the problem of solving likelihood equations by means of a maximum entropy (ME) approach. Unlike standard procedures that require equating the score function of the maximum likelihood problem at zero, we propose an alternative strategy where the score is instead used as an external informative constraint to the maximization of the convex Shannon’s entropy function. The problem involves the reparameterization of the score parameters as expected values of discrete probability distributions where probabilities need to be estimated. This leads to a simpler situation where parameters are searched in smaller (hyper) simplex space. We assessed our proposal by means of empirical case studies and a simulation study, the latter involving the most critical case of logistic regression under data separation. The results suggested that the maximum entropy reformulation of the score problem solves the likelihood equation problem. Similarly, when maximum likelihood estimation is difficult, as is the case of logistic regression under separation, the maximum entropy proposal achieved results (numerically) comparable to those obtained by the Firth’s bias-corrected approach. Overall, these first findings reveal that a maximum entropy solution can be considered as an alternative technique to solve the likelihood equation.

Remote sensing using passive sonar in the ocean is a challenging problem due to variations in the geoacoustic structure of the seabed and unknown source location and strength. One way to improve remote sensing is to perform an optimization for geoacoustic and source parameters. We use a Bayesian maximum entropy (BME) approach with a viscous-grain shearing parameterization for two sediment layers. The statistical optimization provides probability distributions for porosity and thickness of the sediment layers as well as ship speed, closest point of approach, and the source strength for the Wales-Heitmeyer empirical source level spectrum. We use this approach on spectrograms of transiting ships collected on a vertical line array during the 2017 seabed characterization experiment. We compare the resulting parameter distributions from distinct ships as well as previous estimates of geoacoustic values and source properties. This research shows that the BME approach obtains estimates for porosity and source strength that have narrow posterior probability distributions. [Work supported by the Office of Naval Research.]

A maximum entropy approach to adaptive statistical language modelling

Understanding how ensembles of neurons collectively interact will be a key step in developing a mechanistic theory of cognitive processes. Recent progress in multineuron recording and analysis techniques has generated tremendous excitement over the physiology of living neural networks. One of the key developments driving this interest is a new class of models based on the principle of maximum entropy. Maximum entropy models have been reported to account for spatial correlation structure in ensembles of neurons recorded from several different types of data. Importantly, these models require only information about the firing rates of individual neurons and their pairwise correlations. If this approach is generally applicable, it would drastically simplify the problem of understanding how neural networks behave. Given the interest in this method, several groups now have worked to extend maximum entropy models to account for temporal correlations. Here, we review how maximum entropy models have been applied to neuronal ensemble data to account for spatial and temporal correlations. We also discuss criticisms of the maximum entropy approach that argue that it is not generally applicable to larger ensembles of neurons. We conclude that future maximum entropy models will need to address three issues: temporal correlations, higher-order correlations, and larger ensemble sizes. Finally, we provide a brief list of topics for future research.

The variations of the b coefficient in the frequency-magnitude relationship for earthquakes which occurred at Mt. Etna from 01/01/90 to 31/12/92 are analyzed; the completeness threshold for our earthquakes catalogue is M = 2.30. The b values calculated using the Maximum Entropy Principle (MEP) are compared to those obtained by conventional methods of Least Squares (LS) and Maximum Likelihood (ML). All the differences among the b values computed using these methods, and the reasons for these differences, are discussed and examined. In particular, our results show that the b values obtained by MEP are lower than the others calculated using LS and ML; this implies that, on the average, LS and ML underestimate the seismic hazard at Mt. Etna. Moreover, temporal variations of bMEP are more evident than the corresponding ones of bLS and bML; indeed, in some cases the trend of bMEP variation is opposite those of bLS and bML. A significant correlation between temporal variations of the volcanic activity and the b values is evident only if the MEP is used; this means that, if b temporal variations are analyzed in order to detect changes in the volcano dynamics and predict the eruptions, the maximum entropy approach should be preferred. Finally, the observed pattern of bMEP temporal variations with regard to the changes in the volcanic activity is consistent with the hypothesis of a compressive stress field acting on Mt. Etna.

In this paper, we present a maximum entropy (maxent) approach to the fusion of experts opinions, or classifiers outputs, problem. The maxent approach is quite versatile and allows us to express in a clear, rigorous, way the a priori knowledge that is available on the problem. For instance, our knowledge about the reliability of the experts and the correlations between these experts can be easily integrated: Each piece of knowledge is expressed in the form of a linear constraint. An iterative scaling algorithm is used in order to compute the maxent solution of the problem. The maximum entropy method seeks the joint probability density of a set of random variables that has maximum entropy while satisfying the constraints. It is therefore the ”most honest” characterization of our knowledge given the available facts (constraints). In the case of conflicting constraints, we propose to minimise the ”lack of constraints satisfaction” or to relax some constraints and recompute the maximum entropy solution. The maxent fusion rule is illustrated by some simulations.

When analyzing acoustical data, the goal often is to find source strength and characteristics of the environment. Instead of finding point estimates, a more informative approach is to find probability distributions from either a Bayesian approach or a maximum entropy approach. The Bayesian approach relies on prior distributions and an estimate of the noise covariance, while the maximum entropy approach relies on an estimate of a “temperature” derived from analogies with statistical mechanics. Different representations of the temperature are appropriate in different situations. The best choice is related to confidence in the model being used and if the model captures all the relevant physics. Examples of how the choice of temperature impacts the posterior distributions are shown using a toy model and a real-world application. Specifically, the maximum entropy approach is applied to models of transiting ship noise to obtain posterior distributions of source level and porosity of the sediment layer. These examples show that the selection of temperature significantly impacts the posterior distributions. [Work supported by the Office of Naval Research.]

Various seismic deconvolution operators can be determined by estimating a seismic wavelet and subsequently designing an appropriate inverse filter which converts the wavelet to a spike. Seismic wavelets and deconvolution operators must be estimated in a time adaptive sense due to the nonstationarity of the seismic trace. The wavelet estimation methods considered in this paper either use the assumption of a minimum phase wavelet and a random impulse response, or the assumption that the wavelet cepstrum is readily separable from the cepstrum of the seismic trace. The former assumption is required in using the Hilbert transform and Wiener‐Levinson wavelet estimations, while the latter assumption is used in homomorphic deconvolution. These wavelet estimates can be used in the design of multichannel Wiener and Kalman deconvolution operators. Multichannel usage of homomorphic deconvolution can also be implemented through various types of cepstral stacking.The discussion of deconvolution filter design focuses on the problems of filter length, degree of prewhitening, and nonstationarity. In designing time adaptive deconvolution filters, the autocorrelation function can be used to monitor the nonstationarity of the seismic trace. The autocorrelation function, which is used in the computation of least squares inverse filters, can be estimated in an optimum fashion by using the maximum entropy method. Differences between minimum phase Wiener deconvolution and maximum entropy deconvolution become more pronounced for shorter data gates. As a result, the maximum entropy approach is preferred for time adaptive deconvolution.In this paper, the performance of various wavelet estimators and inverse filters is discussed using real and synthetic seismic data. Discussions of homomorphic deconvolution and maximum entropy prediction error filtering are merged with descriptions of conventional approaches to deconvolution.

Typically, the lexicon models used in statistical machine translation systems do not include any kind of linguistic or contextual information, which often leads to problems in performing a correct word sense disambiguation. One way to deal with this problem within the statistical framework is to use maximum entropy methods. In this paper, we present how to use this type of information within a statistical machine translation system. We show that it is possible to significantly decrease training and test corpus perplexity of the translation models. In addition, we perform a rescoring of N-Best lists using our maximum entropy model and thereby yield an improvement in translation quality. Experimental results are presented on the so-called Verbmobil Task.

A new maximum entropy approach has been applied to analyse three types of transient transport experiments. For sawtooth propagation experiments in the ASDEX Upgrade and ECRH power modulation and power-switching experiments in the Wendelstein 7-AS Stellarator, either the time evolution of the temperature perturbation or the phase and amplitude of the modulated temperature perturbation are used as non-linear constraints to the profile to be fitted. Simultaneously, the constraints given by the equilibrium temperature profile for steady-state power balance are fitted. In the maximum entropy formulation, the flattest profile consistent with the constraints is found. It was found that determined from sawtooth propagation was greater than the power balance value by a factor of five in the ASDEX Upgrade. From power modulation experiments, employing the measurements of four modulation frequencies simultaneously, the power deposition profile as well as the profile could be determined. A comparison of the predictions of a time-independent model and a power-dependent model is made. The power-switching experiments show that the profile must change within a millisecond to a new value consistent with the power balance value at the new input power. Neither power deposition broadening due to suprathermal electrons nor temperature or temperature gradient dependences of can explain this observation.

We present an attempt of maximum entropy principle to determine valence quark distributions in the proton at very low resolution scale $Q_0^2$. The initial three valence quark distributions are obtained with limited dynamical information from quark model and QCD theory. Valence quark distributions from this method are compared to the lepton deep inelastic scattering data, and the widely used CT10 and MSTW08 data sets. The obtained valence quark distributions are consistent with experimental observations and the latest global fits of PDFs. Maximum entropy method is expected to be particularly useful in the case where relatively little information from QCD calculation is given.