Improvement of ICA based probability density estimation for pattern recognition

In this paper, we propose a fast and versatile blind source separation including closed-form estimation of sources' probability density functions (PDFs), where the ICA's activation function is automatically adapted to various noise conditions. In the proposed method, closed-form second-order ICA and closed-form PDF estimation are introduced as a computational-cost-efficient preprocessing to extract sources' PDFs. Compared with various type of conventional ICAs, e.g., fixed activation-function type and ML-based type, our proposed algorithm can give a faster and higher convergence. Experimental assessment reveals that the proposed method is versatile for handling non-speech sound sources.

The error estimate of Borgonovo's moment-independent index δi is considered, and it shows that the possible computational complexity of δi is mainly due to the probability density function (PDF) estimate because the PDF estimate is an ill-posed problem and its convergence rate is quite slow. So it reminds us to compute Borgonovo's index using other methods. To avoid the PDF estimate, δi, which is based on the PDF, is first approximatively represented by the cumulative distribution function (CDF). The CDF estimate is well posed and its convergence rate is always faster than that of the PDF estimate. From the representation, a stable approach is proposed to compute δi with an adaptive procedure. Since the small probability multidimensional integral needs to be computed in this procedure, a computational strategy named asymptotic space integration is introduced to reduce a high-dimensional integral to a one-dimensional integral. Then we can compute the small probability multidimensional integral by adaptive numerical integration in one dimension with an improved convergence rate. From the comparison of numerical error analysis of some examples, it can be shown that the proposed method is an effective approach to uncertainty importance measure computation.

The non-line-of-sight (NLOS) propagation caused by intermittently blocking of the direct path between the base station and the mobile station is the main difficulty for mobile user location in cellular wireless communication system. So it is indispensable to investigate the techniques about the NLOS error identification and correction. In this paper, we suggest a robust and accurate approach using probability density function (PDF) estimator to identify the NLOS situation, and then an improved biased Kalman filter which depends on PDF estimator as the nonlinearity weighting coefficient in recursive operation is used to mitigate the NLOS error for time of arrival (TOA) measurements. As the PDF estimator is calculated from actual TOA samples, it is adaptive for the complex practical NLOS environments. The simulation results also indicate that with the less prior information about communication environments, the approach we present has a good location performance even in severe NLOS situations.

The main part of the noise in digital images arises when taking pictures or transmission. There is noise in the images captured by the image sensors of the real world. Noise, based on its causes can have different probability density functions. For example, such a model is called the Poisson distribution function of the random nature of photon arrival process that is consistent with the distribution of pixel values measured. The parameters of the noise probability density function (PDF) can be achieved to some extent the properties of the sensor. But, we need to estimate the parameters for imaging settings. If we assume that the PDF of noise is approximately Gaussian, then we need only to estimate the mean and variance because the Gaussian PDF with only two parameters is determined. In fact, in many cases, PDF of noise is not Gaussian and it has unknown distribution. In this study, we introduce a generalized probability density function for modeling noise in images and propose a method to estimate its parameters. Because the generalized probability density function has multiple parameters, so use common parameter estimation techniques such as derivative method to maximize the likelihood function would be extremely difficult. In this study, we propose the use of evolutionary algorithms for global optimization. The results show that this method accurately estimates the probability density function parameters.

Non-parametric representation of pixel intensity distribution is quite effective to construct proper background model and to detect foreground objects accurately. However, from the viewpoint of practical application, the computation cost of the distribution estimation should be reduced. In this paper, we present fast estimation of the probability density function (PDF) of pixel value using Parzen density estimation and foreground object detection based on the estimated PDF. Here, the PDF is computed by partially updating the PDF estimated at the previous frame, and it greatly reduces the computation cost of the PDF estimation. Thus, the background model adapts quickly to changes in the scene and, therefore, foreground objects can be robustly detected. Several experiments show the effectiveness of our approach.

A new method for the approximation of multivariate scalar probability density functions (PDFs) in turbulent reacting flow by means of a joint presumed discrete distribution (jPDD) is presented. The jPDDs can be generated with specified mean values and variances as well as covariances. Correlations between variables – e.g. fluctuating mixture fractions and/or reaction progress – can thereby be taken into account. In this way the new approach overcomes an important limitation of ordinary presumed PDF methods, where statistical independence between the variables is often assumed. Different methods are presented to generate discrete distributions, based either on biased random number generators or on mixing models familiar from PDF transport models. The new approach is extensively validated on a turbulent flow configuration with simultaneous mixing and reaction. Large eddy simulation data as well as results from a transported PDF model are used for the validation of the jPDD approach. The comparison shows that in particular distributions generated with mixing models are able to predict mean reaction rates accurately. For the configuration considered, the neglect of correlations results in significant underestimation of reaction rates. Moreover it is found that higher statistical moments (e.g. the skewness) can influence reaction rates. The consequences for the generation of jPDDs are discussed. In summary, the new jPDD model has the potential to be significantly more accurate than established presumed PDF methods, because correlations between fluctuating variables can be taken into account. At the same time, the new approach is nearly as efficient as standard presumed PDF formulations, since mean rates are computed in a pre-processing step and stored in look-up tables as a function of the first and second moments of the relevant variables.

In this paper we discuss non-parametric estimation of the probability density function (PDF) of a univariate random variable. This problem has been the subject of a vast amount of scientific literature in many domains: while statisticians are mainly interested in the analysis of the properties of proposed estimators, engineers treat the histogram as a ready-to-use tool for dataset analysis. By considering histogram data as a numerical sequence, a simple PDF estimator is presented in this paper. It is based on basic notions related to the reconstruction of a continuous-time signal from a sequence of samples and it is as accurate as kernel-based estimators, widely adopted in the statistical literature. The major properties of the proposed PDF estimator are discussed and then verified by simulations related to the common case of a normal density function.

The moment-independent global sensitivity (MIGS) analysis can help decision makers to efficiently reduce the uncertainty of model output. This article proposes a novel single-loop simulation (NSLS) method for MIGS analysis. Through an interval discretization strategy, estimation of probability density functions (PDFs) in existing methods is transformed to estimation of the marginal probability of the input, the marginal probability of the output and the joint probability of the inputs–output in NSLS, which avoids the computational complexity and error resulting from PDF estimation in MIGS analysis. To drastically reduce the computational cost of NSLS to estimate the MIGS index, an NSLS-based adaptive kriging (NSLS-AK) method is subsequently developed. In NSLS-AK, a rough global kriging model is first constructed, then this kriging model is adaptively updated until the convergent condition is satisfied for accurately and efficiently predicting the responses of the samples in estimating the MIGS index.

An intensity-modulated Gaussian process {x(t)} is defined as the product of a non-negative stationary random process {σx(t)} and a stationary Gaussian process {z(t)} that is statistically independent of {σx(t)}. When such an intensity-modulated process is the excitation to a linear time-invariant system, the response process {y(t)}, conditioned on the excitation-modulating function σx(t), is a generally nonstationary Gaussian process. This property is used to develop an approximation to the probability density function (PDF) of the system response by assuming that the time-varying conditional response variance σ2y(t), conditioned on the excitation-modulating function σx(t), is governed by the gamma PDF. The resulting response process PDF is characterized by two parameters: the mean and variance of the conditional response variance σ2y(t). An alternative approximation to the response PDF, characterized by these same two parameters, is developed by using a Taylor’s series representation in the variable σ2y of the response PDF conditioned on σ2y. This latter PDF approximation is shown to be identical to the Gram–Charlier series of type A through terms involving the Hermite polynomial of degree 6. The two response PDF approximations are shown to give essentially identical results for values of the coefficient of variation of σ2y of one-half or less. Simple, easily evaluated expressions for the mean and variance of σ2y are derived in terms of measurable quantities by utilizing the assumption that fluctuations in σx(t) occur slowly in comparison with those of z(t).

Estimation of probability density functions based on available data is important problem arising in various fields, such as telecommunications, machine learning, data mining, pattern recognition and computer vision. In this paper, we consider Kernel-based non-parametric density estimation methods and derive formulae for variable kernel density estimation using generalized, elliptic Gaussian kernels. The proposed technique is verified on simulated data.

Multivariate online kernel density estimation with Gaussian kernels

The Parzen-window approach is a well-known technique for estimating probability density functions. This paper introduces a modulated Parzen-windows approach. This approach uses kernels at equidistant samples to obtain a probability density function more efficiently. Experiments on both artificial and real data show that the modulated Parzen-windows approach is more efficient in probability density function estimation, without costly preprocessing or severe loss of accuracy.KeywordsProbability Density FunctionKernel FunctionReduction RatioSmoothing ParameterKernel WidthThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

A function approximation based on an orthonormal wave function expansion in a complex space is derived. Although a probability density function (PDF) cannot always be expanded in an orthogonal series in a real space because a PDF is a positive real function, the function approximation can approximate an arbitrary PDF with high accuracy. It is applied to an actor-critic method of reinforcement learning to derive an optimal policy expressed by an arbitrary PDF in a continuous-action continuous-state environment. A chaos control problem and a PDF approximation problem are solved using the actor-critic method with the function approximation, and it is shown that the function approximation can approximate a PDF well and that the actor-critic method with the function approximation exhibits high performance.

Probability estimation and information principles

A blind source separation algorithm is proposed that is based on minimizing Renyi's mutual information by means of nonparametric probability density function (PDF) estimation. The two-stage process consists of spatial whitening and a series of Givens rotations and produces a cost function consisting only of marginal entropies. This formulation avoids the problems of PDF inaccuracy due to truncation of series expansion and the estimation of joint PDFs in high-dimensional spaces given the typical paucity of data. Simulations illustrate the superior efficiency, in terms of data length, of the proposed method compared to fast independent component analysis (FastICA), Comon's (1994) minimum mutual information, and Bell and Sejnowski's (1995) Infomax.