Parameter estimation and model selection for a class of hysteretic systems using Bayesian inference

In this paper a unified theory is presented for system identification in structural dynamics using recursive frequency-domain filters. Output error, equation error, and instrumental variable (I. V.) filters are developed as special cases. Parameter estimates are extracted from noisy simulated data from a multi-degree-of-freedom system. The equation error filter is strongly convergent but is prone to asymptotic bias error in the presence of measurement noise. An I. V. filter is developed in order to overcome the bias problem while maintaining the desirable convergence properties of the equation error approach.

기존의 frequentist 추론에 비해 Bayesian 추론에서의 가설 검정 및 모형 선택 문제는 학자들 간에 일치된 견해를 보이지 못하고 있으며 아직도 논란이 되는 것들이 많다. Bayesian 추론에서 가설 검정 및 모형 선택의 기준으로 널리 쓰이는 Bayes factor는 이해하기 쉬우나 여러 경우에 구하기 어려운 단점이 존재한다. 그 외에 다른 기준으로 Spiegelhalter 등 (2002)가 제시한 DIC(Deviance Information Criterion)과 frequentist 추론에서의 P-value에 대비되는 Bayesian P-value가 있다. 본 논문에서는 Swiss banknote 자료를 Bayesian 로지스틱 회귀모형으로 분석하고 관련 기준들을 구하여 각 기준들이 일관성 있는 결론을 보이는지 확인하고자 한다. Model selection and hypothesis testing problems in Bayesian inference are still debated between scholars. Bayesian factors traditionally used as a criterion in Bayesian hypothesis testing and model selection, are easy to understand but sometimes hard to compute. In addition, there are other model selection criterions such as DIC(Deviance Information Criterion) by Spiegelhalter et al. (2002) and Bayesian P-values for testing. In this paper, we briefly introduce the Bayesian hypothesis testing and model selection procedure. In addition we have applied a Bayesian inference to Swiss banknote data by a fitting logistic regression model and computing several test statistics to see if they provide consistent results.

Model selection is an important part of any statistical analysis. Many tools are suggested for selecting the best model including frequentist and Bayesian perspectives. There is often a considerable uncertainty in the selection of a particular model to be the best approximating model. Model selection uncertainty arises when the data are used for both model selection and parameter estimation. Bias in estimators of model parameters often arise when data based selection has been done. Therefore, model averaging of the parameter estimators will be done to alleviate the bias in model selection in a set of candidate models, by combining the information from a set of candidate models. This paper is two-fold, new criteria of model selection are proposed based on different averages of AIC, BIC, AICc, and HQC. Also, model averaging is introduced to compare the parameter estimators in model averaging with the ones in model selection. Two Simulation studies are considered, the first is for model selection and showed that the new proposed criteria are lies between some of the known criteria such as AIC, BIC, AICc, and HQC, and so they can be used as new criteria of model selection. The second simulation study is for model averaging and showed that the parameter estimators have less bias and less predicted mean square error (PMSE) compared with the parameter estimators in model selection.

A review and comparison of four commonly used Bayesian and maximum likelihood model selection tools

Capture-Recapture models estimate unknown population sizes. Eight standard closed population models exist, allowing for time, behavioral, and heterogeneity effects. Bayesian versions of these models are presented and use of Akaike's Information Criterion (AIC) and the Deviance Information Criterion (DIC) are explored as model selection tools, through simulation and real dataset analysis.

Only very few failure data can be obtained for the time censored test of high‐reliability and long‐life products. For very few failure data, the current methods fail to obtain both the point estimation and confidence interval of reliability parameters. If the point estimation and confidence interval of reliability parameters are obtained based on different methods, the results tend to be unreliable. In this study, based on the existing research, a Bayesian reliability evaluation method for very few failure data under the Weibull distribution was proposed. First, the range of failure probability was limited based on the convexity and self‐features of the Weibull distribution function. Second, based on the background of the sample with very few failure data, the pretest distribution function and parameters were set and solved. The point estimation and confidence interval model of failure probability based on the Bayesian formula was established. The improved match distribution curve method was used to compute both the point estimation and confidence interval of reliability parameters. Furthermore, by comparing the results of numerical examples, the calculation results obtained by the proposed method were verified as being very reasonable. Finally, taking wet friction plates as an example, the results showed the effectiveness of this method in engineering practice.

The analysis of ecological data has changed in two important ways over the last 15years. The development and easy availability of Bayesian computational methods has allowed and encouraged the fitting of complex hierarchical models. At the same time, there has been increasing emphasis on acknowledging and accounting for model uncertainty. Unfortunately, the ability to fit complex models has outstripped the development of tools for model selection and model evaluation: familiar model selection tools such as Akaike's information criterion and the deviance information criterion are widely known to be inadequate for hierarchical models. In addition, little attention has been paid to the evaluation of model adequacy in context of hierarchical modeling, i.e., to the evaluation of fit for a single model. In this paper, we describe Bayesian cross-validation, which provides tools for model selection and evaluation. We describe the Bayesian predictive information criterion and a Bayesian approximation to the BPIC known as the Watanabe-Akaike information criterion. We illustrate the use of these tools for model selection, and the use of Bayesian cross-validation as a tool for model evaluation, using three large data sets from the North American Breeding Bird Survey.

Mottled Enamel

Bootstrap method for minimum message length autoregressive model order selection

A better estimation of surface reaction efficiency of semiconductor-grade graphite with atomic nitrogen, as well as the calibration error are calculated using Bayesian updating based on experimental data. Compared with a conventional deterministic model, the stochastic model approach is a powerful tool in the sense that the model is capable of taking into account underlying error correlations among the data quantities. In this paper, we investigate four different stochastic models (called “stochastic system model classes” herein) corresponding to different descriptions of modeling and measurement error structures, given one deterministic physical model. These stochastic system model classes differ in the covariance matrix structure that is used in the uncertainty model to represent uncertainties associated with the physical model and experimental measurements. For each model class, Bayesian inference is used to estimate the posterior probabilities of the physical model parameters as well as of the stochastic model parameters. Model comparison and selection are then applied based on two measures including Bayesian evidence and Bayesian information criterion, as well as the deviance information criterion. Both measures suggest the stochastic model class, which considers that a correlation between errors in two data quantities among different data points is the most plausible. With the stochastic model class, the range of uncertainty in surface reaction efficiency is estimated to be about two orders of magnitude at .

This study was conducted to estimate the genetic parameters influencing egg quality in Khorasan Razavi native fowl. (Co)Variance components were estimated by the Bayesian statistical method via Gibbs sampling in GIBBS3F90 software. 1000 eggs (28-29 weeks old) were collected from 775 hens of the ninth generation of Khorasan-Razavi Province native fowl breeding center. External (egg weight,specific gravity, long length, short length, shape index, shell strength, shell weight, shell thickness) and internal (yolk diameter, yolk height, yolk weight, yolk index, albumen diameter, albumen height, albumen weight, albumen index, Haugh unit) egg quality traits were measured. Six univariate animal models were used for estimation of genetic parameters and the best model for each trait was determined by deviance information criterion (DIC). Genetic and phenotypic correlations between traits were estimated using bivariate animal model. Direct heritability estimates ranged from 0.10 (egg width) to 0.39 (yolk index). For all traits except albumen diameter and albumen index, the inclusion of maternal effects in the model resulted in considerable reduction in direct heritability. Genetic correlations of egg weight with shell strength and shell thickness were negative. In conclusion, due to genetic correlations among external and internal egg quality traits, selection based on egg weight and shell thickness may improve external and internal egg quality traits. Also, including maternal effects in the form of genetic or permanent environment in the statistical model resulted in more accurate estimates for most traits.

Over-fitting is a ubiquitous problem in machine learning, and a variety of techniques to avoid over-fitting the training sample have proven highly effective, including early stopping, regularization, and ensemble methods. However, while over-fitting in training is widely appreciated and its avoidance now a standard element of best practice, over-fitting can also occur in model selection. This form of over-fitting can significantly degrade generalization performance, but has thus far received little attention. For example the kernel and regularization parameters of a support vector machine are often tuned by optimizing a cross-validation based model selection criterion. However the cross-validation estimate of generalization performance will inevitably have a finite variance, such that its minimizer depends on the particular sample on which it is evaluated, and this will generally differ from the minimizer of the true generalization error. Therefore if the cross-validation error is aggressively minimized, generalization performance may be substantially degraded. In general, the smaller the amount of data available, the higher the variance of the model selection criterion, and hence the more likely over-fitting in model selection will be a significant problem. Similarly, the more hyper-parameters to be tuned in model selection, the more easily the variance of the model selection criterion can be exploited, which again increases the likelihood of over-fitting in model selection. Over-fitting in model selection is empirically demonstrated to pose a substantial pitfall in the application of kernel learning methods and Gaussian process classifiers. Furthermore, evaluation of machine learning methods can easily be significantly biased unless the evaluation protocol properly accounts for this type of over-fitting. Fortunately the common solutions to avoiding over-fitting in training also appear to be effective in avoiding over-fitting in model selection. Three examples are presented based on regularization of the model selection criterion, early stopping in model selection and minimizing the number of hyper-parameters to be tuned during model selection.

There exist quite a few criteria for penalty-based model selection. Although they have various justifications for large sample problems, their performance under small or moderate sample size is unclear which hinders the development of model combination methods using the appropriate penalty term. In this paper, we assess the performance of seven model selection criteria based on linear regression models with unknown noise variance. We set the true data generation mechanism to be within the model set as well as outside the model set. In the latter case, soft model selection through multiple model fusion is proposed and its difference from Bayesian model averaging is highlighted. The penalty term used in each model selection criterion provides a natural link to estimate the model probability without assuming any prior knowledge of the unknown parameter. An important question is whether the estimated model probabilities are consistent when multiple models are fused for prediction or interpolation. We argue that strong consistency only holds under large sample regime while soft model selection can still be better than choosing a single model with small sample size. Our numerical results using different model selection criteria for polynomial fitting indicate that the conditional model estimator (CME) has the best performance in selecting the correct model order and fusing multiple models for prediction and interpolation. The minimum description length (MDL) based criteria are next to CME and outperform Bayesian information criterion (BIC) and Akaike information criterion (AIC) significantly.

This chapter provides a very brief introduction to Bayesian model selection. The “Survivor Problem” is expanded in this chapter, where the focus is now on comparing two models that predict how long a contestant will last in a game of Survivor: one model uses years of formal education as a predictor, and a second model uses grit as a predictor. Gibbs sampling is used for parameter estimation. Deviance Information Criterion (commonly abbreviated as DIC) is used as a guide for model selection. Details of how this measure is computed are described. The chapter also discusses model assessment (model fit) and Occam’s razor.

In this paper we apply model selection criteria in time series and regression analysis to the estimation of the number of signals in the MUSIC (multiple signal classification) method. We compare the following model selection (information) criteria: AIC, HQ, BIC, AICC, and the recently introduced WIC as the weighted average of AICC and BIC. The general form of the above information criteria consists of a log likelihood function expressed in terms of the eigenvalues of the covariance matrix and a unique penalty term. In our estimation procedure, the number of signals is obtained by minimizing each of the above criteria. A linear antenna array example is presented to compare the performance of the above model selection criteria in signal processing problem.