Non-parametric bayesian measurement noise density estimation in non-linear filtering

Data assimilation is the process of estimating the state of dynamic systems (linear or nonlinear, Gaussian or non-Gaussian) as accurately as possible from noisy observational data. Although the Three Dimensional Variational (3D-VAR) methods, Four Dimensional Variational (4D-VAR) methods and Ensemble Kalman filter (EnKF) methods are widely used and effective for linear and Gaussian dynamics, new methods of data assimilation are required for the general situation, that is, nonlinear non-Gaussian dynamics. General Bayesian recursive estimation theory is reviewed in this thesis. The Bayesian estimation approach provides a rather general and powerful framework for handling nonlinear, non-Gaussian, as well as linear, Gaussian estimation problems. Despite a general solution to the nonlinear estimation problem, there is no closed-form solution in the general case. Therefore, approximate techniques have to be employed. In this thesis, the sequential Monte Carlo (SMC) methods, commonly referred to as the particle filter, is presented to tackle non-linear, non-Gaussian estimation problems. In this thesis, we use the SMC methods only for the nonlinear state estimation problem, however, it can also be used for the nonlinear parameter estimation problem. In order to demonstrate the new methods in the general nonlinear non-Gaussian case, we compare Sequential Monte Carlo (SMC) methods with the Ensemble Kalman Filter (EnKF) by performing data assimilation in nonlinear and non-Gaussian dynamic systems. The models used in this study are referred to as state-space models. The Lorenz 1963 and 1966 models serve as test beds for examining the properties of these assimilation methods when used in highly nonlinear dynamics. The application of Sequential Monte Carlo methods to different fixed parameters in dynamic models is considered. Four different scenarios in the Lorenz 1063 [sic] model and three different scenarios in the Lorenz 1996 model are designed in this study for both the SMC methods and EnKF method with different filter siz

ABSTRACTA natural Bayesian approach for mixture models with an unknown number of components is to take the usual finite mixture model with symmetric Dirichlet weights, and put a prior on the number of components—that is, to use a mixture of finite mixtures (MFM). The most commonly used method of inference for MFMs is reversible jump Markov chain Monte Carlo, but it can be nontrivial to design good reversible jump moves, especially in high-dimensional spaces. Meanwhile, there are samplers for Dirichlet process mixture (DPM) models that are relatively simple and are easily adapted to new applications. It turns out that, in fact, many of the essential properties of DPMs are also exhibited by MFMs—an exchangeable partition distribution, restaurant process, random measure representation, and stick-breaking representation—and crucially, the MFM analogues are simple enough that they can be used much like the corresponding DPM properties. Consequently, many of the powerful methods developed for inference in DPMs can be directly applied to MFMs as well; this simplifies the implementation of MFMs and can substantially improve mixing. We illustrate with real and simulated data, including high-dimensional gene expression data used to discriminate cancer subtypes. Supplementary materials for this article are available online.

Accurate forecasting of photovoltaic (PV) generated electricity is essential for efficiently managing and integrating Renewable Energy (RE) into electricity distribution systems. This research investigation optimizes Feature Selection (FS) and prediction results for PV energy prediction by applying Bayesian Density Estimation (BDE) with Elastic Net (ELNET) regression analysis. This phenomenon and unacceptable outcomes are prevalent when applying conventional regression algorithms on datasets with significant results and addressing predictor multicollinearity. Improved FS and multicollinearity control has been rendered feasible by ELNET, which integrates the best features of Ridge and Lasso regression. ELNET eliminates these challenges through the implementation of L1 and L2 penalties. Non-parametric prediction Bayesian Density Estimation (BDE) is comprehensive data regarding residual distributions and predictor impacts. By incorporating ELNET’s regularisation and FS abilities with BDE’s statistical prediction and adaptability, the recommended ELNET-BDE is proposed to attain more accurate and reliable predictions. This technique has been used to assess massive data sets developing from Visakhapatnam, India, incorporating historical PV energy generation combined with definite Meteorological Factors (MF). Considering comprehensive data preliminary processing, FS, and validation, ELNET-BDE outperforms existing methods. Research investigations demonstrate that the ELNET-BDE model attains significantly lower Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) than contesting Machine Learning (ML) algorithms like Artificial Neural Network (ANN), Support Vector Machine (SVM), Random Forest (RF), and Gradient Boosting Machines (GBM). Compared with distinct FS techniques, RMSE can be minimized by up to 15% and MAE by up to 20%. The findings specify a substantial improvement in accuracy in prediction, emphasizing how the model can be used for improving solar power grid integration and energy for improved RE management.

Nonparametric Density Estimation and Tests of Continuous Time Interest Rate Models

Nonparametric Bayesian estimation of survival function under random left truncation

With the increasing proportion of wind power access in southern China, the impact of the randomness and volatility of wind power generation on power system operation cannot be ignored. Therefore, it is necessary to study the characteristics and random distribution of wind power generation in southern China. Firstly, the influence of time and geographical factors on wind power generation characteristics is analyzed. Secondly, an improved non-parametric kernel density estimation method considering the optimal bandwidth selection of kernel function is proposed to establish the random distribution models of wind power generation in five southern provinces under multiple time scales. The evaluation index system of model fitting effect is also established. Thirdly, the effectiveness of the improved non-parametric kernel density estimation method proposed is verified by comparing with two parameter estimation methods based on typical empirical distribution function. Finally, the differences of probability characteristics of wind power generation in different provinces of southern China are analyzed by daily, quarterly and annual time scales, which can play an auxiliary role in decision-making of power system dispatching, power planning, site selection and capacity determination of wind farms.

To evaluate the impact of the randomness and correlation of photovoltaic (PV) and load on AC/DC systems with a multiport current flow controller (M-CFC), this paper proposes a probabilistic optimal power flow calculation for AC/DC systems based on a nonparametric kernel density estimation. First, according to the M-CFC model, the DC power flow calculation method with M-CFC was deduced, and its influence on line loss was analyzed. Second, a nonparametric kernel density estimation with an adaptive bandwidth was used to accurately describe the probability distribution of the PV and load, and correlation samples of the PV and load were obtained by the mixed copula function. Then an optimization model that considers system loss and static security was established, and a fast nondominated sorting genetic algorithm based on the elite strategy (NSGA-II) was used to calculate the multi-objective probability optimal power flow of the AC/DC system. Finally, a case study was performed on the modified IEEE39 bus system using measured PV and load data. We verified the nonparametric kernel density estimation with an adaptive bandwidth can better adapt to random component uncertainty, and M-CFC can improve the static security of the system.

The aim of this study was to implement a Dirichlet process mixture (DPM) model for automatic tumor edge identification on (18)F-fluorodeoxyglucose positron emission tomography ((18)F-FDG PET) images by optimizing the parameters on which the algorithm depends, to validate it experimentally, and to test its robustness. The DPM model belongs to the class of the Bayesian nonparametric models and uses the Dirichlet process prior for flexible nonparametric mixture modeling, without any preliminary choice of the number of mixture components. The DPM algorithm implemented in the statistical software package R was used in this work. The contouring accuracy was evaluated on several image data sets: on an IEC phantom (spherical inserts with diameter in the range 10-37 mm) acquired by a Philips Gemini Big Bore PET-CT scanner, using 9 different target-to-background ratios (TBRs) from 2.5 to 70; on a digital phantom simulating spherical/uniform lesions and tumors, irregular in shape and activity; and on 20 clinical cases (10 lung and 10 esophageal cancer patients). The influence of the DPM parameters on contour generation was studied in two steps. In the first one, only the IEC spheres having diameters of 22 and 37 mm and a sphere of the digital phantom (41.6 mm diameter) were studied by varying the main parameters until the diameter of the spheres was obtained within 0.2% of the true value. In the second step, the results obtained for this training set were applied to the entire data set to determine DPM based volumes of all available lesions. These volumes were compared to those obtained by applying already known algorithms (Gaussian mixture model and gradient-based) and to true values, when available. Only one parameter was found able to significantly influence segmentation accuracy (ANOVA test). This parameter was linearly connected to the uptake variance of the tested region of interest (ROI). In the first step of the study, a calibration curve was determined to automatically generate the optimal parameter from the variance of the ROI. This "calibration curve" was then applied to contour the whole data set. The accuracy (mean discrepancy between DPM model-based contours and reference contours) of volume estimation was below (1 ± 7)% on the whole data set (1 SD). The overlap between true and automatically segmented contours, measured by the Dice similarity coefficient, was 0.93 with a SD of 0.03. The proposed DPM model was able to accurately reproduce known volumes of FDG concentration, with high overlap between segmented and true volumes. For all the analyzed inserts of the IEC phantom, the algorithm proved to be robust to variations in radius and in TBR. The main advantage of this algorithm was that no setting of DPM parameters was required in advance, since the proper setting of the only parameter that could significantly influence the segmentation results was automatically related to the uptake variance of the chosen ROI. Furthermore, the algorithm did not need any preliminary choice of the optimum number of classes to describe the ROIs within PET images and no assumption about the shape of the lesion and the uptake heterogeneity of the tracer was required.

For binomial data analysis, many methods based on empirical Bayes interpretations have been developed, in which a variance‐stabilizing transformation and a normality assumption are usually required. To achieve the greatest model flexibility, we conduct nonparametric Bayesian inference for binomial data and employ a special nonparametric Bayesian prior—the Bernstein–Dirichlet process (BDP)—in the hierarchical Bayes model for the data. The BDP is a special Dirichlet process (DP) mixture based on beta distributions, and the posterior distribution resulting from it has a smooth density defined on [0, 1]. We examine two Markov chain Monte Carlo procedures for simulating from the resulting posterior distribution, and compare their convergence rates and computational efficiency. In contrast to existing results for posterior consistency based on direct observations, the posterior consistency of the BDP, given indirect binomial data, is established. We study shrinkage effects and the robustness of the BDP‐based posterior estimators in comparison with several other empirical and hierarchical Bayes estimators, and we illustrate through examples that the BDP‐based nonparametric Bayesian estimate is more robust to the sample variation and tends to have a smaller estimation error than those based on the DP prior. In certain settings, the new estimator can also beat Stein's estimator, Efron and Morris's limited‐translation estimator, and many other existing empirical Bayes estimators.The Canadian Journal of Statistics40: 328–344; 2012 © 2012 Statistical Society of Canada

Nonparametric curves occur in a wide variety of contexts, including repeated measurements mean profiles, survivor functions and in different types of smoothing techniques such as nonparametric regression and density estimation. In some cases confidence bands can be attached to these curves as an indication of the variability of estimation. This is more difficult in the case of nonparametric regression and density estimation where bias is present. In the important case of the comparison of two curves, attention can be focused instead on whether there are differences between the curves. In this paper, the idea of a reference band for the comparison of two curves is introduced The band is derived from the standard error of the difference between the two curves at each point. The bands have a simple hypothesis testing interpretation which applies equally well to smoothing methods where bias occurs. It does not remove the need for a global test of effects of interest, but it can be very useful as a graphical means of exploring where any identified differences might lie, or of explaining why apparent differences do not actually contribute strong evidence of statistical significance to the global comparison of curves. Reference bands for equality are derived and explored in a variety of settings. Reference bands for parallelism are also derived for nonparametric regression models. Reference bands for parametric models, against which nonparametric regression and density estimates can be compared, are derived for linearity and for normality.

Unsupervised outlier detection using random subspace and subsampling ensembles of Dirichlet process mixtures

Hidden Markov random field (HMRF) models are widely used for image segmentation, as they appear naturally in problems where a spatially constrained clustering scheme is asked for. A major limitation of HMRF models concerns the automatic selection of the proper number of their states, i.e., the number of region clusters derived by the image segmentation procedure. Existing methods, including likelihood- or entropy-based criteria, and reversible Markov chain Monte Carlo methods, usually tend to yield noisy model size estimates while imposing heavy computational requirements. Recently, Dirichlet process (DP, infinite) mixture models have emerged in the cornerstone of nonparametric Bayesian statistics as promising candidates for clustering applications where the number of clusters is unknown a priori; infinite mixture models based on the original DP or spatially constrained variants of it have been applied in unsupervised image segmentation applications showing promising results. Under this motivation, to resolve the aforementioned issues of HMRF models, in this paper, we introduce a nonparametric Bayesian formulation for the HMRF model, the infinite HMRF model, formulated on the basis of a joint Dirichlet process mixture (DPM) and Markov random field (MRF) construction. We derive an efficient variational Bayesian inference algorithm for the proposed model, and we experimentally demonstrate its advantages over competing methodologies.

Empirical Bayes estimates in exponential reliability model

International audience

Particle filtering, also known as Sequential Monte Carlo methods (SMC), is a sophisticated model estimation techniques based on simulation. Particle filtering has important applications in location, tracking and other fields. It indicates probability using particle set and can be used in any space-state model. Its core idea is to extract a random state from the posterior probability express the distribution. In general, particle filtering is a process which uses a set of stochastic sample propagating in space-state to approximate probability density function and to replace integral operation with mean value of a sample to obtain minimum state variance distribution. It solves the restriction that nonlinear filtering should match Gaussian distribution, expresses a wider range of distribution than Gaussian distribution and has a strong ability to model the nonlinear characteristic of variance parameter. This paper introduces the application of particle filtering in visual tracking. Finally, it puts forward some improved algorithms to revise the inherent deficiencies existing in particle filtering.