Chebyshev moment problems: Maximum entropy and kernel polynomial methods

The authors describe two Chebyshev recursion methods for calculations with very large sparse Hamiltonians, the kernel polynomial method (KPM) and the maximum entropy method (MEM). They are especially applicable to physical properties involving large numbers of eigenstates, which include densities of states, spectral functions, thermodynamics, total energies, as well as forces for molecular dynamics and Monte Carlo simulations. The authors apply Chebyshev methods to the electronic structure of Si, the thermodynamics of Heisenberg antiferromagnets, and a polaron problem.

Kernel Polynomial Approximations for Densities of States and Spectral Functions

Entropy methods enable a convenient general approach to providing a probability distribution with partial information. The minimum cross-entropy principle selects the distribution that minimizes the Kullback–Leibler divergence subject to the given constraints. This general principle encompasses a wide variety of distributions, and generalizes other methods that have been proposed independently. There remains, however, some confusion about the breadth of entropy methods in the literature. In particular, the asymmetry of the Kullback–Leibler divergence provides two important special cases when the target distribution is uniform: the maximum entropy method and the maximum log-probability method. This paper compares the performance of both methods under a variety of conditions. We also examine a generalized maximum log-probability method as a further demonstration of the generality of the entropy approach.

The modeling of complex service systems entails capturing many sub-components of the system, and the dependencies that exist among them in the form of a joint probability distribution. Two common methods for constructing joint probability distributions from experts using partial information include maximum entropy methods and copula methods. In this paper we explore the performance of these methods in capturing the dependence between random variables using correlation coefficients and lower-order pairwise assessments. We focus on the case of discrete random variables, and compare the performance of these methods using a Monte Carlo simulation when the variables exhibit both independence and non-linear dependence structures. We show that the maximum entropy method with correlation coefficients and the Gaussian copula method perform similarly, while the maximum entropy method with pairwise assessments performs better particularly when the variables exhibit non-linear dependence.

This article introduces two new maximum entropy (ME) methods for modeling the distribution of time to an event. One method is within the classical ME framework and provides characterizations of change point models such as the piecewise exponential distribution. The second method uses the entropy of the equilibrium distribution (ED) for the objective function and provides new characterizations of the exponential, Weibull, Pareto, and uniform distributions. With the same moment constraints, the classical ME and the maximum ED entropy algorithms generate different models for the interarrival time. © 2014 Wiley Periodicals, Inc. Naval Research Logistics 61: 427–434, 2014

We propose a probabilistic model for estimating word confidence by fusing predictor features. Starting from the maximum entropy (ME) method, we first prove that ME model is equivalent to the best model with certain form to the minimum expected cross entropy (MECE) criterion. Under the MECE criterion, We extend the form of ME model by introducing a hidden state. We call the new model hidden-state maximum entropy (HSME) model. In a keyword-spotting task, we combine predictor features from both phonetic and word-level systems. Compared to lattice posterior alone, recall at 80% precision is improved from 38.1% to 49.5% on voicemail and from 37.1% to 51.9% on Switchboard. Compared with other fusion methods, HSME consistently outperforms decision tree, and most cases SVM.

The simulation of lattice model systems for quantum materials is one of the most important approaches to understand quantum properties of matter in condensed matter physics. The main task in the simulation is to diagonalize a Hamiltonian matrix for the system and evaluate the electronic density of energy states. Kernel polynomial method (KPM) is one of the promising simulation methods. Because KPM contains a fine-grain recursive part in the algorithm, it is hard to parallelize it under the thread level parallelism such as on a supercomputer or a cluster computer. This paper focuses on methods to parallelize KPM on a massively parallel environment of GPU, aiming to achieve high parallelism for more speedups than the recent CPUs. This paper proposes two implementation methods called the full map and the sliding window methods, and evaluates the performances in the recent GPU platform. To enlarge available simulation sizes and at the same time to enhance the performance, this paper also describes additional optimization techniques depending on the GPU architecture.

Background and Aim:African swine fever (ASF) is an infectious disease and a major viral pig disease that threatens pork production in several locations globally. The mortality rate of ASF in domestic pigs is very high, causing a decrease in pig populations and significant economic losses for farmers. Environmental or ecological risk factors are the most important associated with the spread of the ASF virus. Environmental (or ecological) niche models are commonly used to estimate the probability of an event using the maximum entropy (Maxent) method. This study aimed to estimate the probability risk of future ASF outbreaks in North Sumatra, Indonesia.Materials and Methods:Secondary data from the National Animal Health System Database (iSIKHNAS), including data on the ASF outbreaks of 2019–2020 in North Sumatra, Indonesia, were used in this study. The first analysis performed involved the identification of environmental risk factors using multiple regression analysis. The second analysis performed was the estimation of probability risk for future ASF outbreaks in North Sumatra, Indonesia, using the Maxent method. Data processing was performed using Microsoft Excel, ArcGIS version 10.5 software (ESRI, California, United States), Maxent version 3.4.4 software, and Rstudio (http://www.r-project.org/).Results:The Maxent method was found to be highly accurate with a statistically significant area under the curve value of 0.860. The greatest contributing environmental factor identified by the model was the harbor, which contributed 57%. The range of high probability risk of future ASF outbreaks was found to be 0.723–0.84.Conclusion:The estimation of the highest probability risk of future ASF outbreaks in North Sumatra, Indonesia, was 0.723–0.84. The most contributing environmental factor identified using the Maxent method was harbors, at 57%. This methodology can be used to carry out subsequent ASF analyses and contribute to developing prevention and control strategies in this area.

A computational framework for a moment-based O(N) tight-binding atomistic method is presented, analyzed, and applied to the problem of electronic properties of deformed carbon nanotubes, where N is the number of atoms in the system. The moment-based approach is based on the maximum entropy and kernel polynomial methods for constructing the electronic density of states from local statistical information about the environment around individual atoms. Random-walk statistics are formally presented as the basis for several methods to collect the moments of the density of states in a computationally efficient manner. The computational complexity and accuracy of these methods are systematically analyzed. Using these methods for the problem of deformed carbon nanotubes, it is shown that the computational cost for some cases, per atom, scales as efficiently as O(M log M), where M is the desired number of moments in the expansion of the density of states. These methods are compared to other methods such as direct diagonalization and a Green's function approach.

Radio polarization images of the jets of Active Galactic Nuclei (AGN) can provide a deep insight into the launching and collimation mechanisms of relativistic jets. However, even at VLBI scales, resolution is often a limiting factor in the conclusions that can be drawn from observations. The maximum entropy method (MEM) is a deconvolution algorithm that can outperform the more common <scp>clean</scp> algorithm in many cases, particularly when investigating structures present on scales comparable to or smaller than the nominal beam size with ‘super-resolution’. A new implementation of the MEM suitable for single- or multiple-wavelength VLBI polarization observations has been developed and is described here. Monte Carlo simulations comparing the performances of <scp>clean</scp> and MEM at reconstructing the properties of model images are presented; these demonstrate the enhanced reliability of MEM over <scp>clean</scp> when images of the fractional polarization and polarization angle are constructed using convolving beams that are appreciably smaller than the full <scp>clean</scp> beam. The results of using this new MEM software to image VLBA observations of the AGN 0716+714 at six different wavelengths are presented, and compared to corresponding maps obtained with <scp>clean</scp>. MEM and <scp>clean</scp> maps of Stokes <it>I</it>, the polarized flux, the fractional polarization and the polarization angle are compared for convolving beams ranging from the full <scp>clean</scp> beam down to a beam one-third of this size. MEM's ability to provide more trustworthy polarization imaging than a standard <scp>clean</scp>-based deconvolution when convolving beams appreciably smaller than the full <scp>clean</scp> beam are used is discussed.

Comparison of maximum entropy and filtered back-projection methods to reconstruct rapid-scan EPR images

We present a detailed study of the applications of two stochastic approaches, stochastic optimization method (SOM) and stochastic analytical inference (SAI), to extract spectral functions from Euclidean correlation functions. SOM has the advantage that it does not require prior information. On the other hand, SAI is a more generalized method based on Bayesian inference. Under mean field approximation SAI reduces to the often-used maximum entropy method (MEM), and for a specific choice of the prior SAI becomes equivalent to SOM. To test the applicability of these two stochastic methods to lattice QCD, firstly, we apply these methods to various reasonably chosen model correlation functions, and present detailed comparisons of the reconstructed spectral functions obtained from SOM, SAI and MEM. Next, we present similar studies for charmonia correlation functions obtained from lattice QCD computations using clover-improved Wilson fermions on large, fine, isotropic lattices at $0.75$ and $1.5T_c$, $T_c$ being the deconfinement transition temperature of a pure gluon plasma. We find that SAI and SOM give consistent results to MEM at these two temperatures.

Many fields of physics use quantum Monte Carlo techniques, but struggle to estimate dynamic spectra via the analytic continuation of imaginary-time quantum Monte Carlo data. One of the most ubiquitous approaches to analytic continuation is the maximum entropy method (MEM). We supply a dual Newton optimization algorithm to be used within the MEM and provide analytic bounds for the algorithm's error. The optimization algorithm is freely availible on github (https://github.com/MPF-Optimization-Laboratory/DualPerspective.jl). The MEM is typically used with Bryan's controversial algorithm [Rothkopf, "Bryan’s Maximum Entropy Method" Data 5.3 (2020)]. We present new theoretical issues that are not yet in the literature. Our algorithm has all the theoretical benefits of Bryan's algorithm without these theoretical issues the implementation of the dual Newton optimizer within the MEM is freely available on github (https://github.com/chunatho/InverseProblemSolvers). We compare the MEM with Bryan's optimization to the MEM with our dual Newton optimization on test problems from lattice quantum chromodynamics and plasma physics. These comparisons show that in the presence of noise the dual Newton algorithm produces better estimates and error bars; this indicates the limits of Bryan's algorithm's applicability. We use the MEM to investigate authentic quantum Monte Carlo data for the uniform electron gas at warm dense matter conditions and further substantiate the roton-type feature in the dispersion relation.

The maximum entropy (ME) method is a powerful supervised machine learning technique that is useful for various tasks. In this paper, we introduce new studies that successfully employ ME for natural language processing (NLP) problems including machine translation and information extraction. Specifically, we demonstrate, using simulation results, three applications of ME for NLP: estimation of categories, extraction of important features, and correction of error data items. We also evaluate the comparative performance of the proposed ME methods with other state-of-the-art approaches.

Most optimization methods for logistic regression or maximum entropy solve the primal problem. They range from iterative scaling, coordinate descent, quasi-Newton, and truncated Newton. Less efforts have been made to solve the dual problem. In contrast, for linear support vector machines (SVM), methods have been shown to be very effective for solving the dual problem. In this paper, we apply coordinate descent methods to solve the dual form of logistic regression and maximum entropy. Interestingly, many details are different from the situation in linear SVM. We carefully study the theoretical convergence as well as numerical issues. The proposed method is shown to be faster than most state of the art methods for training logistic regression and maximum entropy.