Empirical Bayes estimates in exponential reliability model

As there has been a paradigm shift in the learning load from a human subject to a computer, machine learning has been considered as a useful tool for Brain-Computer Interfaces (BCIs). In this paper, we propose a novel Bayesian framework for discriminative feature extraction for motor imagery classification in an EEG-based BCI in which the class-discriminative frequency bands and the corresponding spatial filters are optimized by means of the probabilistic and information-theoretic approaches. In our framework, the problem of simultaneous spatiospectral filter optimization is formulated as the estimation of an unknown posterior probability density function (pdf) that represents the probability that a single-trial EEG of predefined mental tasks can be discriminated in a state. In order to estimate the posterior pdf, we propose a particle-based approximation method by extending a factored-sampling technique with a diffusion process. An information-theoretic observation model is also devised to measure discriminative power of features between classes. From the viewpoint of classifier design, the proposed method naturally allows us to construct a spectrally weighted label decision rule by linearly combining the outputs from multiple classifiers. We demonstrate the feasibility and effectiveness of the proposed method by analyzing the results and its success on three public databases.

In this paper, we propose a fast and versatile blind source separation including closed-form estimation of sources' probability density functions (PDFs), where the ICA's activation function is automatically adapted to various noise conditions. In the proposed method, closed-form second-order ICA and closed-form PDF estimation are introduced as a computational-cost-efficient preprocessing to extract sources' PDFs. Compared with various type of conventional ICAs, e.g., fixed activation-function type and ML-based type, our proposed algorithm can give a faster and higher convergence. Experimental assessment reveals that the proposed method is versatile for handling non-speech sound sources.

In this paper, we introduce a novel independent component analysis (ICA) algorithm, which is truly blind to the particular underlying distribution of the mixed signals. Using a nonparametric kernel density estimation technique, the algorithm performs simultaneously the estimation of the unknown probability density functions of the source signals and the estimation of the unmixing matrix. Following the proposed approach, the blind signal separation framework can be posed as a nonlinear optimization problem, where a closed form expression of the cost function is available, and only the elements of the unmixing matrix appear as unknowns. We conducted a series of Monte Carlo simulations, involving linear mixtures of various source signals with different statistical characteristics and sample sizes. The new algorithm not only consistently outperformed all state-of-the-art ICA methods, but also demonstrated the following properties: 1) Only a flexible model, capable of learning the source statistics, can consistently achieve an accurate separation of all the mixed signals. 2) Adopting a suitably designed optimization framework, it is possible to derive a flexible ICA algorithm that matches the stability and convergence properties of conventional algorithms. 3) A nonparametric approach does not necessarily require large sample sizes in order to outperform methods with fixed or partially adaptive contrast functions.

The non-line-of-sight (NLOS) propagation caused by intermittently blocking of the direct path between the base station and the mobile station is the main difficulty for mobile user location in cellular wireless communication system. So it is indispensable to investigate the techniques about the NLOS error identification and correction. In this paper, we suggest a robust and accurate approach using probability density function (PDF) estimator to identify the NLOS situation, and then an improved biased Kalman filter which depends on PDF estimator as the nonlinearity weighting coefficient in recursive operation is used to mitigate the NLOS error for time of arrival (TOA) measurements. As the PDF estimator is calculated from actual TOA samples, it is adaptive for the complex practical NLOS environments. The simulation results also indicate that with the less prior information about communication environments, the approach we present has a good location performance even in severe NLOS situations.

The error estimate of Borgonovo's moment-independent index δi is considered, and it shows that the possible computational complexity of δi is mainly due to the probability density function (PDF) estimate because the PDF estimate is an ill-posed problem and its convergence rate is quite slow. So it reminds us to compute Borgonovo's index using other methods. To avoid the PDF estimate, δi, which is based on the PDF, is first approximatively represented by the cumulative distribution function (CDF). The CDF estimate is well posed and its convergence rate is always faster than that of the PDF estimate. From the representation, a stable approach is proposed to compute δi with an adaptive procedure. Since the small probability multidimensional integral needs to be computed in this procedure, a computational strategy named asymptotic space integration is introduced to reduce a high-dimensional integral to a one-dimensional integral. Then we can compute the small probability multidimensional integral by adaptive numerical integration in one dimension with an improved convergence rate. From the comparison of numerical error analysis of some examples, it can be shown that the proposed method is an effective approach to uncertainty importance measure computation.

Blind detection in symmetric non-Gaussian noise with unknown PDF using maximum entropy method with moment generating function constraints

Multivariate online kernel density estimation with Gaussian kernels

Probability density function (PDF) estimation is a fundamentally important problem for statistical pattern recognition. Independent component analysis (ICA) can be applied to the feature vectors so that the PDF estimation of a high dimensional vector can be converted to the PDF estimation of several 1-dimensional variables. However in practice we find that this PDF is in poor generalization ability for pattern classification because of the implied noise. Hence, this paper proposes an improvement of ICA based PDF estimation method. A latent variable model is built to separate the noise from the feature vector so that the pattern information and the noise can be dealt with respectively. Based on the latent variable model, a modified ICA based PDF is deduced. The validity of our proposed method is demonstrated by the experiments of off-line handwritten numeral recognition.

A blind source separation algorithm is proposed that is based on minimizing Renyi's mutual information by means of nonparametric probability density function (PDF) estimation. The two-stage process consists of spatial whitening and a series of Givens rotations and produces a cost function consisting only of marginal entropies. This formulation avoids the problems of PDF inaccuracy due to truncation of series expansion and the estimation of joint PDFs in high-dimensional spaces given the typical paucity of data. Simulations illustrate the superior efficiency, in terms of data length, of the proposed method compared to fast independent component analysis (FastICA), Comon's (1994) minimum mutual information, and Bell and Sejnowski's (1995) Infomax.

Since in many life-testing situations it is not unlikely to note the unpredictable fluctuation of the scale parameter in a failure model, it is justifiable to consider such a parameter as a random variable and, thus, appeal to a Bayesian analysis. In specific, let 0 denote the random variable associated with the scale parameter and 0 its realization. Obviously, a Bayesian analysis depends on the utilization of prior information which, in this case, we assume to exist either in the form of a prior distribution of 0 or a sequence of sufficient statistics from past experiments. For the ordinary Bayes approach, we appeal to a well-known transformation to generalize the results of Bhattacharya (1967) for the one-parameter exponential model so as to include the flexibility provided by the shape parameter ? in the Weibull distribution. In fact, the Weibull failure model has an increasing ( > 1) or decreasing ( < 1) failure rate and, thus, is likely to describe the life-span of items with variable failure rates. The empirical Bayes estimation technique was largely motivated by Robbins (1955), who assumed the existence of a prior distribution for an unknown parameter but not the knowledge of its form. Instead, he substitutes past information which he assumes to exist as a result of the repetitive nature in the problem of estimation. Thus, in the absence of knowledge concerning the form of the prior distribution, we appeal to an empirical Bayes approach to estimate the scale parameter. By using this estimate, an estimate of the reliability function is made possible.

Many statistic based machine learning methods depend on the estimation of probability density function from observations. Non-parametric density estimation algorithms based on minimizing expirical risk using support vector machine (SVM) are quite general and powerful, but have a significant disadvantage in the smoothness of estimation result. In this paper, we studies the vicinal risk minimization based estimation algorithm, and propose a new construction algorithm of vicinity function. Experiments are carried out which prove that the performance of new algorithm is obviously improved.

Estimation of probability density functions based on available data is important problem arising in various fields, such as telecommunications, machine learning, data mining, pattern recognition and computer vision. In this paper, we consider Kernel-based non-parametric density estimation methods and derive formulae for variable kernel density estimation using generalized, elliptic Gaussian kernels. The proposed technique is verified on simulated data.

Four methods are discussed for determining the smoothing parameters in an adaptive exponential smoothing model which is used to assess reliability of a complex system tested in stages. The adaptive model is defined as R_I = &#x003B1;_Ir_I + (1 - &#x003B1;_I)R_i-1, i = 2,...k, where R<sub.i-1</sub. is the assessed reliability at stage (i -1), ri is the ratio of the number of successes to the number of trials at stage i. Among the four procedures given here for determining the smoothing parameters &#x003B1;_I two empirical methods and an empirical Bayes procedure are considered in some detail including a numerical example in which these techniques are compared.

&amp;lt;p&amp;gt;Applications of the Bayesian statistics require specifying a prior distribution for each unknown parameter to be estimated. The commonly used definition of a Bayesian prior distribution, information about an uncertain parameter, does not provide guidance on how to derive and formulate a prior distribution. In practice, we often use &amp;quot;non-informative&amp;quot; priors or priors based on mathematical convenience. I present a normative definition of the prior based on the shared features of the James-Stein estimator, the empirical Bayes method, and the Bayesian hierarchical model. I use the word &amp;quot;normative&amp;quot; to mean &amp;quot;prescriptive&amp;quot;. It also reflects the meaning that the definition can be inconsistent with one another insofar as different types of parameters. I present two case studies where this definition guided me to formulate the modeling processes: one on modeling and predicting cyanobacterial toxin concentration in Lake Erie using chlorophyll-a concentrations (Lake Erie example) and the other on improving the accuracy of calibration-curve-based chemical measurement method (calibration-curve example). The Lake Erie example illustrates temporal exchangeable units, while the calibration-curve example showcases the ubiquity of such exchangeable units.&amp;lt;/p&amp;gt;

This article discusses the Bayesian approach for count data using non-homogeneous Poisson processes, considering different prior distributions for the model parameters. A Bayesian approach using Markov Chain Monte Carlo (MCMC) simulation methods for this model was first introduced by [1], taking into account software reliability data and considering non-informative prior distributions for the parameters of the model. With the non-informative prior distributions presented by these authors, computational difficulties may occur when using MCMC methods. This article considers different prior distributions for the parameters of the proposed model, and studies the effect of such prior distributions on the convergence and accuracy of the results. In order to illustrate the proposed methodology, two examples are considered: the first one has simulated data, and the second has a set of data for pollution issues at a region in Mexico City.