BC-DMKDE: Bayesian Classifier Based on Distance-Corrected Multivariate Kernel Density Estimation

A Bayesian approach to bandwidth selection for multivariate kernel density estimation

Density estimation is an important aspect of statistics. Statistical inference often requires the knowledge of observed data density. A common method of density estimation is the kernel density estimation (KDE). It is a nonparametric estimation approach which requires a kernel function and a window size (smoothing parameter H). It aids density estimation and pattern recognition. So, this work focuses on the use of a modified intersection of confidence intervals (MICIH) approach in estimating density. The Nigerian crime rate data reported to the Police as reported by the National Bureau of Statistics was used to demonstrate this new approach. This approach in the multivariate kernel density estimation is based on the data. The main way to improve density estimation is to obtain a reduced mean squared error (MSE), the errors for this approach was evaluated. Some improvements were seen. The aim is to achieve adaptive kernel density estimation. This was achieved under a sufficiently smoothing technique. This adaptive approach was based on the bandwidths selection. The quality of the estimates obtained of the MICIH approach when applied, showed some improvements over the existing methods. The MICIH approach has reduced mean squared error and relative faster rate of convergence compared to some other approaches. The approach of MICIH has reduced points of discontinuities in the graphical densities the datasets. This will help to correct points of discontinuities and display adaptive density. Keywords: approach, bandwidth, estimate, error, kernel density

Kernel density estimation is a well known method involving a smoothing parameter (the bandwidth) that needs to be tuned by the user. Although this method has been widely used, the bandwidth selection remains a challenging issue in terms of balancing algorithmic performance and statistical relevance. The purpose of this paper is to study a recently developed bandwidth selection method, called Penalized Comparison to Overfitting (PCO). We first provide new theoretical guarantees by proving that PCO performed with non-diagonal bandwidth matrices is optimal in the oracle and minimax approaches. PCO is then compared to other usual bandwidth selection methods (at least those which are implemented in the R-package) for univariate and also multivariate kernel density estimation on the basis of intensive simulation studies. In particular, cross-validation and plug-in criteria are numerically investigated and compared to PCO. The take home message is that PCO can outperform the classical methods without algorithmic additional cost.

This paper presents fabrication of complex surficial micro-features employing a cross-innovative hybrid process inspired from lithography and Jet-ECM. The process is referred here as mask electrolyte jet machining (MEJM). MEJM is a non-contact machining process which combines high resolution of lithography and greater flexibility of Jet-ECM. It is a non-contact process which can fabricate variety of microstructures on difficult-to-machine materials without need of expensive tooling. The presented work demonstrates the process performance of this technology by statistical analysis and multivariate kernel density estimation (KDE) based on probabilistic density function. Micro-letters are fabricated as an example of complex surficial structure comprising of multiple intersecting, straight and curved grooves. The processing response is characterized in terms of geometrical size, similarity ratio, and cumulative shape deviation. Experimental results demonstrated that micro letters with good repeatability (minimum SD of shape error ratio 0.297%) and shape accuracy (minimum shape error of 0.039%) can be fabricated with this technology. The results suggest MEJM could be a promising technology for batch manufacturing of surface microstructures with high productivity.

A novel maximum a posteriori (MAP) method for dynamic single-photon emission computed tomography image reconstruction is proposed. The prior probability is modeled as a multivariate kernel density estimator (KDE), effectively modeling the prior probability non-parametrically, with the aim of reducing the effects of artifacts arising from inconsistencies in projection measurements in low-count regimes where projections are dominated by noise. The proposed prior spatially and temporally limits the variation of time-activity functions (TAFs) and attracts similar TAFs together. The similarity between TAFs is determined by the spatial and range scaling parameters of the KDE-like prior. The resulting iterative image reconstruction method is evaluated using two simulated phantoms, namely the extended cardiac-torso (XCAT) heart phantom and a simulated Mini-Deluxe Phantom. The phantoms were chosen to observe the effects of the proposed prior on the TAFs based on the vicinity and abutments of regions with different activities. Our results show the effectiveness of the proposed iterative reconstruction method, especially in low-count regimes, which provides better uniformity within each region of activity, significant reduction of spatiotemporal variations caused by noise, and sharper separation between different regions of activity than expectation maximization and an MAP method employing a more traditional Gibbs prior.

Nonparametric estimation for a probability density function that describes multivariate data has typically been addressed by kernel density estimation (KDE). A novel density estimator recently developed by Farmer and Jacobs offers an alternative high-throughput automated approach to univariate nonparametric density estimation based on maximum entropy and order statistics, improving accuracy over univariate KDE. This article presents an extension of the single variable case to multiple variables. The univariate estimator is used to recursively calculate a product array of one-dimensional conditional probabilities. In combination with interpolation methods, a complete joint probability density estimate is generated for multiple variables. Good accuracy and speed performance in synthetic data are demonstrated by a numerical study using known distributions over a range of sample sizes from 100 to 106 for two to six variables. Performance in terms of speed and accuracy is compared to KDE. The multivariate density estimate developed here tends to perform better as the number of samples and/or variables increases. As an example application, measurements are analyzed over five filters of photometric data from the Sloan Digital Sky Survey Data Release 17. The multivariate estimation is used to form the basis for a binary classifier that distinguishes quasars from galaxies and stars with up to 94% accuracy.

K-means is the best known clustering algorithm, because of its usage simplicity, fast speed and efficiency.However, resultant clusters are influenced by the randomly selected initial centroids.Therefore, many techniques have been implemented to solve the mentioned issue. In this paper, a new version of the k-means clustering algorithm named as ImpKmeans shortly (An Improved Version of K-Means Algorithm by Determining Optimum InitialCentroids Based on Multivariate Kernel Density Estimation and Kd-tree) that uses kernel density estimation, to find the optimum initial centroids, is proposed.Kernel density estimation is used, because it is a nonparametric distribution estimation method, that can identify density regions.To understand the efficiency of the ImpKmeans, we compared it with some state-of-the-art algorithms.According to the experimental studies, the proposed algorithm was better than the compared versions of k-means.While ImpKmeans was the most successful algorithm in 46 tests of 60, the second-best algorithm, was the best on 34 tests.Moreover, experimental results indicated that the ImpKmeans is fast, compared to the selected k-means versions.

Although density‐based clustering algorithms can successfully define clusters in arbitrary shapes, they encounter issues if the dataset has varying densities or neck‐typed clusters due to the requirement for precise distance parameters, such as eps parameter of DBSCAN. These approches assume that data density is homogenous, but this is rarely the case in practice. In this study, a new clustering algorithm named ANDClust (Adaptive Neighborhood Distance‐based Clustering Algorithm) is proposed to handle datasets with varying density and/or neck‐typed clusters. The algorithm consists of three parts. The first part uses Multivariate Kernel Density Estimation (MulKDE) to find the dataset's peak points, which are the start points for the Minimum Spanning Tree (MST) to construct clusters in the second part. Lastly, an Adaptive Neighborhood Distance (AND) ratio is used to weigh the distance between the data pairs. This method enables this approach to support inter‐cluster and intra‐cluster density varieties by acting as if the distance parameter differs for each data of the dataset. ANDClust is tested on synthetic and real datasets to reveal its efficiency. The algorithm shows superior clustering quality in a good run‐time compared to its competitors. Moreover, ANDClust could effectively define clusters of arbitrary shapes and process high‐dimensional, imbalanced datasets may have outliers.

With the advent of the Legacy Survey of Space and Time, time-domain astronomy will be faced with an unprecedented volume and rate of data. Real-time processing of variables and transients detected by such large-scale surveys is critical to identifying the more unusual events and allocating scarce follow-up resources efficiently. We develop an algorithm to identify these novel events within a given population of variable sources. We determine the distributions of magnitude changes (dm) over time intervals (dt) for a given passband f, , and use these distributions to compute the likelihood of a test source being consistent with the population or being an outlier. We demonstrate our algorithm by applying it to the DECam multiband time-series data of more than 2000 variable stars identified by Saha et al. in the Galactic Bulge that are largely dominated by long-period variables and pulsating stars. Our algorithm discovers 18 outlier sources in the sample, including a microlensing event, a dwarf nova, and two chromospherically active RS CVn stars, as well as sources in the blue horizontal branch region of the color–magnitude diagram without any known counterparts. We compare the performance of our algorithm for novelty detection with the multivariate Kernel Density Estimator and Isolation Forest on the simulated PLAsTiCC data set. We find that our algorithm yields comparable results despite its simplicity. Our method provides an efficient way for flagging the most unusual events in a real-time alert-broker system.

Publication in the conference proceedings of EUSIPCO, Lisbon, Portugal, 2014

A metabolomics approach to characterise and identify various Mycobacterium species

In a broad and fundamental type of ‘inverse problems’ in science, one infers a spatially distributed physical attribute based on observations of processes that are controlled by the spatial attribute in question. The data-generating field processes, known as ‘forward processes’, are usually nonlinear with respect to the spatial attribute, and are often defined non-analytically by a numerical model. The data often contain a large number of elements with significant inter-correlation. We propose a general statistical method to tackle this problem. The method is centered on a parameterization device called ‘anchors’ and an iterative algorithm for deriving the distribution of anchors conditional on the observed data. The algorithm draws upon techniques of importance sampling and multivariate kernel density estimation with weighted samples. Anchors are selected automatically; the selection evolves in iterations in a way that is tailored to important features of the attribute field. The method and the algorithm are general with respect to the scientific nature and technical details of the forward processes. Conceptual and technical components render the method in contrast to standard approaches that are based on regularization or optimization. Some important features of the proposed method are demonstrated by examples from the earth sciences, including groundwater flow, rainfall-runoff and seismic tomography.

With the rapid increase in distributed wind generation, considerable efforts have been devoted to the microgrid day-ahead scheduling. The effectiveness of those methods will highly depend on the selection of the uncertainty sets. We propose a distribution-free approach for wind power scenario generation, using sequence generative adversarial networks. To capture the temporal correlation, the model adopts the long short-term memory architecture and uses generative adversarial networks coupled with reinforcement learning, which, in contrast to the existing methods, avoids manual labeling and captures the complex dynamics of the weather. We conduct case studies based on the data from the Bonneville Power Administration and the National Renewable Energy Laboratory, and show that the generated scenarios can better characterize the variability of wind power and reduce the risk of uncertainties, compared with those produced by Gaussian distribution, vanilla long short-term memory, and multivariate kernel density estimation. Moreover, the proposed method achieves better performance when applied to the day-ahead scheduling of microgrids.

A variable bandwidth selector in multivariate kernel density estimation

Opinion mining has become a major part in today's economy. People would want to know more about a product and the customers opinion before buying it. Companies would also want to know the opinions of the customers. Therefore, analyzing the customer’s opinion is important. A new customer would consider a product as good by analyzing the opinions of other customers. The opinions are collected from various areas, which include blogs, web forums, and product review sites. Classifying these large set of opinions requires a good classifier. In view of this, a comparative study of three classification techniques - Naive Bayes classifier with Kernel Density Estimation (KDE), Support Vector Machine (SVM), Decision Tree and KNN was made. To evaluate the classifier accuracy, precision, recall and F-measure techniques are used. Experimental results show that the Naive Bayes with Kernel Density Estimation (KDE) classifier achieved higher accuracy among others.