Global approximation to arbitrary cost functions: A Bayesian approach with application to US banking

A Markov Chain Monte Carlo technique for parameter estimation and inference in pesticide fate and transport modeling

The Ideal Observer (IO) performance has been advocated when optimizing medical imaging systems for signal detection tasks. However, analytical computation of the IO test statistic is generally intractable. To approximate the IO test statistic, sampling-based methods that employ Markov-Chain Monte Carlo (MCMC) techniques have been developed. However, current applications of MCMC techniques have been limited to several object models such as a lumpy object model and a binary texture model, and it remains unclear how MCMC methods can be implemented with other more sophisticated object models. Deep learning methods that employ generative adversarial networks (GANs) hold great promise to learn stochastic object models (SOMs) from image data. In this study, we described a method to approximate the IO by applying MCMC techniques to SOMs learned by use of GANs. The proposed method can be employed with arbitrary object models that can be learned by use of GANs, thereby the domain of applicability of MCMC techniques for approximating the IO performance is extended. In this study, both signal-known-exactly (SKE) and signal-known-statistically (SKS) binary signal detection tasks are considered. The IO performance computed by the proposed method is compared to that computed by the conventional MCMC method. The advantages of the proposed method are discussed.

This article estimates technical and allocative inefficiencies and increase in costs therefrom of individual firms using a translog cost system consisting of the cost function and the cost share equations. We call it a nonlinear random-effects system because technical and allocative inefficiencies are random (hence the term random effects) and are separated from the random noise terms appearing in each equation of the system, and because the inefficiency terms appear in the system in a highly nonlinear fashion, which helps in separating them from random errors. We use Bayesian inference procedures based on Markov chain Monte Carlo (MCMC) techniques to estimate the proposed system. Inferences on firm-specific technical inefficiency and both input-specific and firm-specific allocative inefficiencies are developed using MCMC techniques. We apply the new methods to a sample of 500 U.S. commercial banks. We focus on input allocation problem based on the assumption that banks minimize cost. Empirical results show that cost for the top (bottom) 10% of banks is increased by at least 3% (11%) due to technical inefficiency. In contrast, very few banks are found to be efficient in allocating all the inputs. Costs are increased by 13% on average due to input misallocation. Increase in costs due to both technical and allocative inefficiencies for the top (bottom) 10% of the banks is at least 11% (29%). When translated into dollar figures, this result indicates that elimination of technical and allocative inefficiencies would save the top (bottom) 10% of the banks more than $$.20 ($$3.56) million. We also find that none of the banks in our sample exceeded the efficient scale size, although most of them were operating near their optimum scale (unitary returns to scale).

Using the standard one-zone synchrotron self-Compton model and the Markov chain Monte Carlo (MCMC) technique, we systematically analyse the quasi-simultaneous multiwavelength spectral energy distributions (SEDs) of Mrk 421 and Mrk 501 during states of relatively low activity. With this model in place, a semi-analytical method is developed to examine the uncertainty in jet power estimation caused by degeneracy of the radiative models. The semi-analytical method, in combination with the MCMC technique, allows us to explore the jet properties over a wide range of the variability time-scale. Our results seem to support the following. (i) In both Mrk 421 and Mrk 501, the jets are powered by rapidly rotating black holes. The black hole spin in Mrk 501 could be lower than that in Mrk 421 or possibly they are equal under the assumption of the Blandford–Znajek mechanism. (ii) The energy losses, which could be used to form the large-scale radio structure, are important for reconciling the differences of the kinetic power derived from the observations of the large-scale structure and the SED fitting results. Moreover, the jet energy losses in the propagation are more significant for Mrk 501 than for Mrk 421.

Group Importance Sampling for particle filtering and MCMC

Appropriate models in biostatistics are often quite complicated. Such models are typically most easily fit using Bayesian methods, which can often be implemented using simulation techniques. Markov chain Monte Carlo (MCMC) methods are an important set of tools for such simulations. We give an overview and references of this rapidly emerging technology along with a relatively simple example. MCMC techniques can be viewed as extensions of iterative maximization techniques, but with random jumps rather than maximizations at each step. Special care is needed when implementing iterative maximization procedures rather than closed-form methods, and even more care is needed with iterative simulation procedures: it is substantially more difficult to monitor convergence to a distribution than to a point. The most reliable implementations of MCMC build upon results from simpler models fit using combinations of maximization algorithms and noniterative simulations, so that the user has a rough idea of the location and scale of the posterior distribution of the quantities of interest under the more complicated model. These concerns with implementation, however, should not deter the biostatistician from using MCMC methods, but rather help to ensure wise use of these powerful techniques.

We investigate the estimation of multi-stress strength reliability under progressively first failure censoring for the generalized inverted exponential distribution using classical and Bayesian approaches. Classical estimation employs maximum likelihood estimators with the Expectation-Maximization algorithm to handle incomplete data. Bayesian estimation utilizes Markov Chain Monte Carlo (MCMC) techniques and Tierney-Kadane's approximation to approximate the posterior distribution. Furthermore, A generalized entropy loss function is employed to measure estimation accuracy. Finally, Validation is provided through Monte Carlo simulations and real-world applications. Findings highlight the robustness and accuracy of Bayesian methods, particularly with informative priors, and underscore the strengths of MCMC techniques and Bayesian Credible Intervals in specific aspects of estimation.

Count variables are often positively skewed and may include many zero observations, requiring specific statistical approaches. Interpreting abiotic factor changes in insect populations of crop pests, under this condition, can be difficult. The analysis becomes even more complicated because of possible temporal or spatial correlation, irregularly spaced data, heterogeneity over time, and zero inflation. Generalized additive models (GAM) are important tools to evaluate abiotic factors. Moreover, Markov chain Monte Carlo (MCMC) techniques can be used to fit a model that contains a temporal correlation structure, based on Bayesian statistics (BGAM). We compared methods of modeling the effects of temperature, precipitation, and time for the Brevicoryne brassicae (L.) population in Uberlândia, Brasil. We applied the proposed BGAM to the data, comparing this to the GAM model with and without autocorrelation for time, using the statistical programming language R. Analysis of deviance identified significant effects of the smoothers for precipitation and time on the frequentist models. With BGAM, the problem in variance estimations for precipitation and temperature from the previous models was solved. Furthermore, trace and density plots for population-level effects for all parameters converged well. The estimated smoothing curves showed a linear effect with an increase of precipitation, where lower precipitation indicated no presence of the aphid. The average temperature did not affect the aphid incidence. Autocorrelation was solved with ARMA structures, and the excess of zero was solved with zero-inflation models. The example of B. brassicae incidence showed how well abiotic (and biotic) factors can be modeled and analyzed using BGAM.

Bayesian inference has been successfully applied in fields as varied as anti-spam filtering, DNA sequencing, war codebreaking and election forecasting. Founded on the apparently simple Bayes’ theorem, which relates the previous distribution of a parameter with its distribution after evidence is collected, Bayesian tools allow for incorporating all existing knowledge about the phenomenon under study in order to improve parameter estimation. Because of the stochastic nature of the wireless channel, Bayesian inference is particularly well suited to the problem of symbol detection in many modern digital communication systems. When combined with Markov Chain Monte Carlo (MCMC) techniques, Bayesian receivers are capable of achieving minimum Bit Error Rate (BER) while avoiding the prohibitively high computational complexity associated with standard Maximum Likelihood (ML) or Maximum A Posteriori (MAP) estimators. In addition, such receivers are capable of numerically integrating out channel coefficients and noise variance, thus avoiding the need to use sub-optimal estimates of these parameters. This tutorial presents the rudiments of Bayesian statistics and MCMC in general, and discusses their applications in wireless communications in particular. The paper also details the design of Bayesian MCMC receiver in a system employing BPSK and ubject frequency-selective fading and Gaussian noise. Afterwards, recent advances in Bayesian receivers are surveyed for several important practical wireless transmission schemes, including MIMO, CDMA and OFDM. In addition, the paper addresses the application of Bayesian tools in challenging channel conditions — namely, nonlinear, non-Gaussian, underwater and fast fading channels.

Bayesian model updating has received considerable attention and has been extensively used in structural damage detection. It provides a rigorous statistical framework for realizing structural system identification and characterizing uncertainties associated with modeling and measurements. The Markov Chain Monte Carlo (MCMC) is a promising tool for inferring the posterior distribution of model parameters to avoid the intractable evaluation of multi-dimensional integration. However, the efficacy of most MCMC techniques suffers from the curse of parameter dimension, which restricts the application of Bayesian model updating to the damage detection of large-scale systems. In addition, there are several MCMC techniques that require users to properly choose application-specific models, based on the understanding of algorithm mechanisms and limitations. As seen in the literature, there is a lack of comprehensive work that investigates the performances of various MCMC algorithms in their application of structural damage detection. In this study, the Differential Evolutionary Adaptive Metropolis (DREAM), a multi-chain MCMC, is explored and adapted to Bayesian model updating. This paper illustrates how DREAM is used for model updating with many uncertainty parameters (i.e., 40 parameters). Furthermore, the study provides a tutorial to users who may be less experienced with Bayesian model updating and MCMC. Two advanced single-chain MCMC algorithms, namely, the Delayed Rejection Adaptive Metropolis (DRAM) and Transitional Markov Chain Monte Carlo (TMCMC), and DREAM are elaborately introduced to allow practitioners to understand better the concepts and practical implementations. Their performances in model updating and damage detection are compared through three different engineering applications with increased complexity, e.g., a forty-story shear building, a two-span continuous steel beam, and a large-scale steel pedestrian bridge.

This paper presents an efficient method for estimating capacity-fade uncertainty in lithium-ion batteries (LIBs) in order to integrate them into the battery-management system (BMS) of electric vehicles, which requires simple and inexpensive computation for successful application. The study uses the pseudo-two-dimensional (P2D) electrochemical model, which simulates the battery state by solving a system of coupled nonlinear partial differential equations (PDEs). The model parameters that are responsible for electrode degradation are identified and estimated, based on battery data obtained from the charge cycles. The Bayesian approach, with parameters estimated by probability distributions, is employed to account for uncertainties arising in the model and battery data. The Markov Chain Monte Carlo (MCMC) technique is used to draw samples from the distributions. The complex computations that solve a PDE system for each sample are avoided by employing a polynomial-based metamodel. As a result, the computational cost is reduced from 5.5 h to a few seconds, enabling the integration of the method into the vehicle BMS. Using this approach, the conservative bound of capacity fade can be determined for the vehicle in service, which represents the safety margin reflecting the uncertainty.

Performance modelling of web applications involves the task of estimating service demands of requests at physical resources, such as CPUs. In this paper, we propose a service demand estimation algorithm based on a Markov Chain Monte Carlo (MCMC) technique, Gibbs sampling. Our methodology is widely applicable as it requires only queue length samples at each resource, which are simple to measure. Additionally, since we use a Bayesian approach, our method can use prior information on the distribution of parameters, a feature not always available with existing demand estimation approaches. The main challenge of Gibbs sampling is to efficiently evaluate the conditional expression required to sample from the posterior distribution of the demands. This expression is shown to be the equilibrium solution of a multiclass closed queueing network. We define a novel approximation to efficiently obtain the normalising constant to make the cost of its evaluation acceptable for MCMC applications. Experimental evaluation based on simulation data with different model sizes demonstrates the effectiveness of Gibbs sampling for service demand estimation.

This paper deals with the classical and Bayesian estimation of multicomponent stress-strength reliability through assuming the exponentiated inverse Rayleigh distribution. Assuming that both stress and strength variates are follows to exponentiated inverse Rayleigh distribution with common and known shape parameter. The multicomponent stress-strength reliability of a system is obtained by the methods of maximum likelihood and Bayesian approach. The results are compared using Markov Chain Monte Carlo (MCMC) technique for both small and large samples. Finally, two data sets of coating weights of iron sheets are analyzed for illustrative purposes.

A novel variational Bayesian method for variable selection in logistic regression models

Precise radial velocity measurements have led to the discovery of ~100\nextrasolar planetary systems. We investigate the uncertainty in the orbital\nsolutions that have been fit to these observations. Understanding these\nuncertainties will become more and more important as the discovery space for\nextrasolar planets shifts to longer and longer periods. While detections of\nshort period planets can be rapidly refined, planets with long orbital periods\nwill require observations spanning decades to constrain the orbital parameters\nprecisely. Already in some cases, multiple distinct orbital solutions provide\nsimilarly good fits, particularly in multiple planet systems. We present a\nmethod for quantifying the uncertainties in orbital fits and addressing\nspecific questions directly from the observational data rather than relying on\nbest fit orbital solutions. This Markov chain Monte Carlo (MCMC) technique has\nthe advantage that it is well suited for the high dimensional parameter spaces\nnecessary for the multiple planet systems. We apply the MCMC technique to\nseveral extrasolar planetary systems, assessing the uncertainties in orbital\nelements for several systems. Our MCMC simulations demonstrate that for some\nsystems there are strong correlations between orbital parameters and/or\nsignificant non-Gaussianities in parameter distributions, even though the\nmeasurement errors are nearly Gaussian. Once these effects are considered the\nactual uncertainties in orbital elements can be significantly larger or smaller\nthan the published uncertainties. We also present simple applications of our\nmethods such as predicting the times of possible transits for GJ 876.\n