A generalized maxentropic inversion procedure for noisy data

The classical maximum entropy (ME) approach to estimating the unknown parameters of a multinomial discrete choice problem, which is equivalent to the maximum likelihood multinomial logit (ML) estimator, is generalized. The generalized maximum entropy (GME) model includes noise terms in the multinomial information constraints. Each noise term is modeled as the mean of a finite set of a priori known points in the interval [−1,1] with unknown probabilities where no parametric assumptions about the error distribution are made. A GME model for the multinomial probabilities and for the distributions associated with the noise terms is derived by maximizing the joint entropy of multinomial and noise distributions, under the assumption of independence. The GME formulation reduces to the ME in the limit as the sample grows large or when no noise is included in the entropy maximization. Further, even though the GME and the logit estimators are conceptually different, the dual GME model is related to a gener...

Entropy methods enable a convenient general approach to providing a probability distribution with partial information. The minimum cross-entropy principle selects the distribution that minimizes the Kullback–Leibler divergence subject to the given constraints. This general principle encompasses a wide variety of distributions, and generalizes other methods that have been proposed independently. There remains, however, some confusion about the breadth of entropy methods in the literature. In particular, the asymmetry of the Kullback–Leibler divergence provides two important special cases when the target distribution is uniform: the maximum entropy method and the maximum log-probability method. This paper compares the performance of both methods under a variety of conditions. We also examine a generalized maximum log-probability method as a further demonstration of the generality of the entropy approach.

This paper demonstrates a robust method for achieving disaggregation in the estimation of flexible-form farm-level multi-input production functions using minimally-specified data sets. Since our ultimate goal is to address important questions related to the distributional effects of policy changes, we place emphasis on the ability of the model to reproduce the characteristics of the existing production system and to predict the outcomes of these changes at a high level of disaggregation. Achieving this requires the use of farm-level models that are estimated across a wide spectrum of sizes and types, which is often difficult to do with traditional econometric methods, due to limitations of data. The approach to estimating flexible-form production functions used in this paper overcomes these limitations, and also avoids the problems that frequently hinder the application of budget-based representative farm models to these type of analyses namely, that of poor calibration to observed behavior. In our estimation procedure, we use a two-stage approach that first generates a set of observation-specific shadow values for incompletely priced inputs, such as irrigation water or family labor, which are used in the second stage, along with the nominal input prices, to produce estimates of crop-specific production functions using Generalized Maximum Entropy (GME) methods. These functions are able to capture the individual heterogeneity of the local production environment, while still allowing the production function to replicate the input usage and outputs produced in the sample data. Since we are able to generate demand, supply, and substitution elasticities, a wide range of policy responses can be modeled. Our paper demonstrates this methodology through an empirical application to Mexico, drawing from a small set of cross-section data collected in the northern Rio Bravo regions. The estimates show that there is considerable heterogeneity in the behavioral response of farmer households of different sizes, both in terms of the returns to scale, as well as in the elasticities of substitution and derived demands for water. Compared to the aggregate-level estimation, we obtain much more accurate and informative policy response behavior, when shocks are imposed on the model.

The overall objective of this research is to analyze the impact of wage and bonus increases on enlisted personnel as well as personnel behavior over time and sensitivity to the macroeconomic conditions. Two types of models are used: the Amortized Cost of Leaving (ACOL) maximum likelihood and the non-ACOL information theoretic Generalized Maximum Entropy (GME) method. The bonus experiments for the ACOL models are done in the traditional manner and a way to improve those is discussed. The experiments for the non-ACOL model are developed and studied here.

Predicting multinomial choices using maximum entropy

The modeling of complex service systems entails capturing many sub-components of the system, and the dependencies that exist among them in the form of a joint probability distribution. Two common methods for constructing joint probability distributions from experts using partial information include maximum entropy methods and copula methods. In this paper we explore the performance of these methods in capturing the dependence between random variables using correlation coefficients and lower-order pairwise assessments. We focus on the case of discrete random variables, and compare the performance of these methods using a Monte Carlo simulation when the variables exhibit both independence and non-linear dependence structures. We show that the maximum entropy method with correlation coefficients and the Gaussian copula method perform similarly, while the maximum entropy method with pairwise assessments performs better particularly when the variables exhibit non-linear dependence.

This article introduces two new maximum entropy (ME) methods for modeling the distribution of time to an event. One method is within the classical ME framework and provides characterizations of change point models such as the piecewise exponential distribution. The second method uses the entropy of the equilibrium distribution (ED) for the objective function and provides new characterizations of the exponential, Weibull, Pareto, and uniform distributions. With the same moment constraints, the classical ME and the maximum ED entropy algorithms generate different models for the interarrival time. © 2014 Wiley Periodicals, Inc. Naval Research Logistics 61: 427–434, 2014

Multicollinearity hampers empirical econometrics. The remedies proposed to date suffer from pitfalls of their own. The ridge estimator is not generally accepted as a vital alternative to the ordinary least-squares (OLS) estimator because it depends upon unknown parameters. The generalized maximum entropy (GME) estimator of Golan, Judge and Miller depends upon subjective exogenous information that affects the estimated parameters in an unpredictable way. This paper presents novel maximum entropy estimators inspired by the theory of light that do not depend upon any additional information. Monte Carlo experiments show that they are not affected by any level of multicollinearity and dominate OLS uniformly. The Leuven estimators are consistent and asymptotically normal.

There was proximity between natural and social science in the history.It couldn't be say that, this proximity continues now.Because sciencis were divided into specialist areas.Scholar who lived even in the middle age or new ages could have profound knowledge in the field of social sciences and natural sciences.While explaining social science scholars that could benefit from the basic information in the field of natürel science.Social science concepts can be intangible.At this point it may be necessary to explain some concepts concretization.It makes it easier to understand and permanence of the affects if we tell you a thing or terminology by concretizing.For this, social science was used from natural science terminology that time to time.We could give some example for this hypothesis.An example of relationship between sociology and biology or political science and sociology-biology-physics which was particularly marked.Especially middle age and eastern scientists have put forward opinions about relationship between social and natural science.For example, Ibn Khaldun, likened the individual to the state.These smilarities has established relevance between science of political and science of biology.On the other side the west scholars have demonstrated many examples of ideas or hypotheses that relationship in new age.

LINPRO: Linear inverse problem library for data contaminated by statistical noise

We propose a probabilistic model for estimating word confidence by fusing predictor features. Starting from the maximum entropy (ME) method, we first prove that ME model is equivalent to the best model with certain form to the minimum expected cross entropy (MECE) criterion. Under the MECE criterion, We extend the form of ME model by introducing a hidden state. We call the new model hidden-state maximum entropy (HSME) model. In a keyword-spotting task, we combine predictor features from both phonetic and word-level systems. Compared to lattice posterior alone, recall at 80% precision is improved from 38.1% to 49.5% on voicemail and from 37.1% to 51.9% on Switchboard. Compared with other fusion methods, HSME consistently outperforms decision tree, and most cases SVM.

The objective of this paper is to evaluate the degree of financial integration achieved in the European Union based on covered interest parity and using Generalized Maximum Entropy. EU countries are divided into two groups according to their current situation with respect to the adoption of the euro. Financial integration before the adoption of the euro is analyzed for the countries that adopted the euro in 1999. Similarly, current financial integration is evaluated for non-euro EU countries. Besides the importance of comparing the situation of the non-euro EU countries with the situation of the euro EU countries previous to the euro adoption, which may be useful to evaluate an eventual decision of the non-euro members to adopt the euro, it is interesting to analyze the performance of Generalized Maximum Entropy. Generalized Maximum Entropy has the ability to estimate the parameters of a regression model without imposing any constrains on the probability distribution of errors and it is robust even when we have ill-posed problems. Overall our results suggest that the degree of financial integration on non-euro countries is lower than the degree of financial integration that existed among euro adopting countries before the adoption of the euro.

Understanding the reciprocal interactions between humans and marine ecosystems has several fundamental difficulties, in particular compatible methodologies and different analytical scales. The issue of scale is central, as the scales chosen for studies of marine systems and human interactions can constrain recognition of the drivers and responses of these systems to global changes. The essential task is to discover how to combine social and natural science scale analyses to understand the impact of natural systems on people and the impact of people on natural systems. We identify characteristic spatial, temporal and organizational scales in marine ecosystems and human interactions, and the difficulties inherent in their cross‐disciplinary application. An approach is suggested focusing on communities of fish and fishers that makes explicit: (1) the need to manage marine resources in such a way as to encompass global to local scales; (2) recognition of the complementary nature of organizational scales between the natural and social sciences and use of appropriate natural science scales in the development of management policies; (3) the need to be aware of shifting temporal baselines and the representative nature of the data over time, for both social and natural sciences; and (4) caution regarding predictive models when humans are included. In terms of methodologies, good scale matches occur across large‐scale social and natural science models and surveys, but problems remain in small‐scale qualitative social studies and in cross‐scale studies. Cumulative case studies appear to provide the best approach, although ‘integrating up’ remains a challenge. Natural and social scientists need to work together to identify these issues of ecosystem processes and human interactions, and their appropriate scales.

Interdisciplinary research within the field of sustainability studies often faces incompatible ontological assumptions deriving from natural and social sciences. The importance of this fact is often underrated and sometimes leads to the wrong strategies. We distinguish between two broad approaches in interdisciplinarity: unificationism and pluralism. Unificationism seeks unification and perceives disciplinary boundaries as conventional, representing no long-term obstacle to progress, whereas pluralism emphasizes more ephemeral and transient interdisciplinary connections and underscores the autonomy of the disciplines with respect to one another. Both approaches have their merits and pitfalls. Unification runs the risk of scientific imperialism, while pluralism can result in insurmountable barriers between disciplines. We made a comparison of eight distinct interdisciplinary attempts at integration of knowledge across social and natural sciences. The comparison was carried out as four pairwise comparisons: environmental economics versus ecological economics, environmental history versus historical ecology, resilience theory versus political ecology, and socio-biology versus actor-network theory. We conclude by showing that none of these prominent eight interdisciplinary fields in and of itself manages to provide, in a satisfactory way, such an integrated understanding of sustainability. We argue for pluralism and advocate complex ways of articulating divergent ontological assumptions. This is not equivalent to pursuing knowledge unification either through scientific imperialism or by catering to the requirements of narrow practical utility. It means prioritizing interdisciplinary integration by simultaneously acknowledging the role of societal and natural factors in accounting for sustainability issues.

Two Chebyshev recursion methods are presented for calculations with very large sparse Hamiltonians, the kernel polynomial method (KPM) and the maximum entropy method (MEM). They are applicable to physical properties involving large numbers of eigenstates such as densities of states, spectral functions, thermodynamics, total energies for Monte Carlo simulations and forces for tight binding molecular dynamics. this paper emphasizes efficient algorithms.