The IOC algorithm

This paper proposes an approach based on fuzzy rough set theory to improve nearest neighbor based classification. Six measures are introduced to evaluate the quality of the nearest neighbors. This quality is combined with the frequency at which classes occur among the nearest neighbors and the similarity w.r.t. the nearest neighbor, to decide which class to pick among the neighbor's classes. The importance of each aspect is weighted using optimized weights. An experimental study shows that our method, Quality, Frequency and Similarity based Fuzzy Nearest Neighbor (QFSNN), outperforms state-of-the-art nearest neighbor classifiers.

The Nearest Neighbor (NN) classifier is one of the most used and well-known techniques for classification. However, NN suffers from several drawbacks such as high storage requirements, low efficiency in classification response, and noise sensitivity in the neighborhood. Prototype weighting as one the effective learning methods, assigns a weight to each prototype in order to determine its importance and simultaneously removes both redundant and noisy ones as a side effect. NN has an instance-based structure, and this is why; it is one of the rare methods for which the results of cross validation can be achieved for all possible combinations called as Complete-Cross-Validation (CCV) without need to construct and train exponential number of models. Using CCV as the objective function in learning the weights of prototypes can increase generalization capability of the classifier and also decrease the randomness of the learning method. Nevertheless, it cannot be computed such efficiently to be used as a fitness function during most of learning algorithms. In this paper, an iterative learning method has been proposed, in which the objective is incrementally improved without computing CCV even once. The proposed approach was assessed through a series of experiments with UCI in comparison with basic NN and related prototype weighting scheme and could outperform them.

The nearest neighbor classifier (NNC) is a non-parametric classification technique that classifies a test pattern to the class of its nearest neighbor in the training data. In this research, we applied the NNC to a standard letter recognition data and obtained a superior classification rate in comparison to extant approaches. A prime drawback to the NNC technique has been the relative inefficiency of the model. A modified NNC was implemented and applied to the same recognition problem. It is found that if we choose a suitable threshold minimum difference for classification, we can reduce the CPU time by half without lowering the performance of the classifier.

Machine learning (ML) applications have been thriving recently, largely attributed to the increasing availability of data. However, inconsistency and incomplete information are ubiquitous in real-world datasets, and their impact on ML applications remains elusive. In this paper, we present a formal study of this impact by extending the notion of Certain Answers for Codd tables, which has been explored by the database research community for decades, into the field of machine learning. Specifically, we focus on classification problems and propose the notion of "Certain Predictions" (CP) --- a test data example can be certainly predicted (CP'ed) if all possible classifiers trained on top of all possible worlds induced by the incompleteness of data would yield the same prediction. We study two fundamental CP queries: (Q1) checking query that determines whether a data example can be CP'ed; and (Q2) counting query that computes the number of classifiers that support a particular prediction (i.e., label). Given that general solutions to CP queries are, not surprisingly, hard without assumption over the type of classifier, we further present a case study in the context of nearest neighbor (NN) classifiers, where efficient solutions to CP queries can be developed --- we show that it is possible to answer both queries in linear or polynomial time over exponentially many possible worlds. We demonstrate one example use case of CP in the important application of "data cleaning for machine learning (DC for ML)." We show that our proposed CPClean approach built based on CP can often significantly outperform existing techniques, particularly on datasets with systematic missing values. For example, on 5 datasets with systematic missingness, CPClean (with early termination) closes 100% gap on average by cleaning 36% of dirty data on average, while the best automatic cleaning approach BoostClean can only close 14% gap on average.

A method to speed up the Bayes classifier

Currently, Nearest-Neighbor approaches (NN) have been widely applied to real world image data mining. These approaches have the following three disadvantages: (i) the performance is inferior on small datasets; (ii) the performance of approximated nearest neighbor search will degrade for data with high dimensions; (iii) they are heavily dependent on the chosen feature and distance measure. To overcome these intrinsic weaknesses, we propose a novel Nearest-Neighbor method, which improves the original NN approaches from three aspects. Firstly, we propose a novel neighborhood similarity measure, where the similarity between test images and labeled images in the database is calculated jointly by the original image-to-image similarity and the average similarity of their neighboring unlabeled data. Secondly, we adopt the kernelized locality sensitive hashing to effectively conduct the nearest neighbor search for high dimensional data. Finally, to enhance the robustness of the method on different genres of images, we propose to fuse the discrimination power of different features by considering all the retrieved nearest neighbors via hashing systems using different features/kernels. Experimental result shows the advantage over traditional Nearest-Neighbor methods using the labeled data only. Even when the ratio of labeled data is very small, our method could also achieve remarkable results, thanks to the help of unlabeled data and multiple features.

Passive polarization based imaging is a useful tool in computer vision and pattern recognition. A passive polarization imaging system forms a polarimetric image from the reflection of ambient light that contains useful information for computer vision tasks such as object detection (classification) and recognition. Applications of polarization based pattern recognition include material classification and automatic shape recognition. In this paper, we present two target detection algorithms for images captured by a passive polarimetric imaging system. The proposed detection algorithms are based on Bayesian decision theory. In these approaches, an object can belong to one of any given number classes and classification involves making decisions that minimize the average probability of making incorrect decisions. This minimum is achieved by assigning an object to the class that maximizes the a posteriori probability. Computing a posteriori probabilities requires estimates of class conditional probability density functions (likelihoods) and prior probabilities. A Probabilistic neural network (PNN), which is a nonparametric method that can compute Bayes optimal boundaries, and a -nearest neighbor (KNN) classifier, is used for density estimation and classification. The proposed algorithms are applied to polarimetric image data gathered in the laboratory with a liquid crystal-based system. The experimental results validate the effectiveness of the above algorithms for target detection from polarimetric data.

Forensic human identification with targeted microbiome markers using nearest neighbor classification

Nearest neighbor classifiers are a popular method for multiclass recognition in a wide range of computer vision and pattern recognition domains. At the same time, the accuracy of nearest neighbor classi?ers is sensitive to the choice of distance measure. This paper introduces an algorithm that uses boosting to learn a distance measure for multiclass k-nearest neighbor classification. Given a family of distance measures as input, AdaBoost is used to learn a weighted distance measure, that is a linear combination of the input measures. The proposed method can be seen both as a novel way to learn a distance measure from data, and as a novel way to apply boosting to multiclass recognition problems that does not require output codes. In our approach, multiclass recognition of objects is reduced to a single binary recognition task, defined on triples of objects. Preliminary experiments with eight UCI datasets yield no clear winner among our method, boosting using output codes, and k-nn classification using an unoptimized distance measure. Our algorithm did achieve lower error rates in some of the datasets, which indicates that it is a method worth considering for nearest neighbor recognition in various pattern recognition domains.

Combining multiple classifiers is an effective technique for improving accuracy. There are many general combining algorithms, such as Bagging, Boosting, or Error Correcting Output Coding, that significantly improve classifiers like decision trees, rule learners, or neural networks. Unfortunately, t hese combining methods do not improve the nearest neighbor classifier. In this paper, we present MFS, a combining algorithm designed to improve the accuracy of the nearest neighbor (NN) classifier. MFS combines multiple NN classifiers each using only a random subset of features. The experimental results are encouraging: On 25 datasets from the UCI repository, MFS significantly outperformed several standard NN variants and was competitive with boosted decision trees. In additional experiments, we show that MFS is robust to irrelevant features, and is able to reduce both bias and variance components of error.

A fully automated method for coronary calcification detection from non-contrast-enhanced, ECG-gated multi-slice computed tomography (CT) data is presented. Candidates for coronary calcifications are extracted by thresholding and component labeling. These candidates include coronary calcifications, calcifications in the aorta and in the heart, and other high-density structures such as noise and bone. A dedicated set of 64 features is calculated for each candidate object. They characterize the object's spatial position relative to the heart and the aorta, for which an automatic segmentation scheme was developed, its size and shape, and its appearance, which is described by a set of approximated Gaussian derivatives for which an efficient computational scheme is presented. Three classification strategies were designed. The first one tested direct classification without feature selection. The second approach also utilized direct classification, but with feature selection. Finally, the third scheme employed two-stage classification. In a computationally inexpensive first stage, the most easily recognizable false positives were discarded. The second stage discriminated between more difficult to separate coronary calcium and other candidates. Performance of linear, quadratic, nearest neighbor, and support vector machine classifiers was compared. The method was tested on 76 scans containing 275 calcifications in the coronary arteries and 335 calcifications in the heart and aorta. The best performance was obtained employing a two-stage classification system with a k-nearest neighbor (k-NN) classifier and a feature selection scheme. The method detected 73.8% of coronary calcifications at the expense of on average 0.1 false positives per scan. A calcium score was computed for each scan and subjects were assigned one of four risk categories based on this score. The method assigned the correct risk category to 93.4% of all scans.

Recognizing aspects of articulation from audio recordings of speech is an important problem, either as an end in itself or as part of an articulatory approach to automatic speech recognition. In this paper we study the frame-level classification of a set of articulatory features (AFs) inspired by the vocal tract variables of articulatory phonology. We compare k nearest neighbor (k-NN) classifiers and multilayer perceptrons (MLPs), using different acoustic feature vectors, and classify the AFs either independently or jointly. We also consider using the MLP outputs for all of the AFs as inputs to k-NN classifiers for the individual AFs, effectively using the MLPs as a form of nonlinear dimensionality reduction and allowing the decision for each AF to be based on the MLPs for the other AFs. We find that MLPs outperform k-NN classifiers, while k-NN classifiers using MLP outputs outperform both. Index Terms: articulatory modeling, speech inversion, multilayer perceptron, k nearest neighbors

The characteristics of society in choosing contraceptive methods are also the crucial factors for the government to prepare the family planning services needed at Bulakamba District, Brebes Regency, Central Java. In this case, a classification process needs to be done to assist the process of classifying the characteristics of society in the selection of contraceptive methods. Multinomial Logistic Regression classification is good in exploring data information meanwhile Fuzzy K Nearest Neighbor (FK-NN) classification is good for handling big data and noise. These two methods used in this study because they are relevant to the data applied and will be compared their classification accuracy through APER and Press's Q calculations.The classification accuracy results obtained based on the APER calculation for Multinomial Logistic Regression is 88,25% and Fuzzy K Nearest Neighbor (FK-NN) is 88,92%. Meanwhile, the Press's Q value of both methods are 9600,945 and 9518,014 greater than χ 2𝛼,1 which is 3,841, so that it is statistically accurate. Based on the results obtained, it can be concluded that Multinomial Logistic Regression classification method has a better classification accuracy than Fuzzy K Nearest Neighbor (FK-NN) method.

Olympian Dreams

In this paper, two novel classifiers, based on locally nearest neighborhood rules, called nearest neighbor line (NNL) and nearest neighbor plane (NNP), are presented for face recognition. The underlying idea of both classifiers is the local linear combination technique that has been previously used in locally linear embedding (LLE) for nonlinear dimension reduction. In comparison to other linear combination based classifiers, such as the nearest feature line (NFL) and the nearest feature plane (NFP), the proposed method has a much lower computation cost. Furthermore, the experimental results on the ORL face database have shown that the performance of both proposed methods are competitive to the NFL and NFP in face classification.