Learning invariant and uniformly distributed feature space for multi-view generation

Data annotation for training of supervised learning algorithms has been a very costly procedure. The aim of deep active learning methodologies is to acquire the highest performance in supervised deep learning models by annotating as few data points as possible. As the feature space of data grows, the application of linear models in active learning settings has become insufficient. Therefore, Deep Bayesian Active Learning methodology which represents model uncertainty has been widely studied. In this paper, a study has been conducted in order to increase the performance of Deep Bayesian Active Learning method. Feature dimensionality reduction is performed on data set by using Variational Autoencoder model. Low dimensional data is used to train a Bayesian Multi Layer Perceptron. The proposed method outperformed the Bayesian Multi Layer Perceptron model which is trained on entire feature space in terms of accuracy performance. The accuracy of the proposed method is tested on baseline datasets.

In this paper, a method of combining fuzzy decision tree with optimal fuzzy-valued feature subset selection (OFFSS) is proposed. This algorithm can first select a feature subset from the entire feature space, and then constructs the fuzzy decision tree on the selected feature subset. This work conducts some experiments to compare the proposed algorithm with fuzzy-ID3. The experiment results show that fuzzy decision trees on feature subset are superior to that on entire feature space in terms of speed and accuracy for classification.

There has been growing interest in the AI community for precise uncertainty quantification. Conditional density models f(y|x), where x represents potentially high-dimensional features, are an integral part of uncertainty quantification in prediction and Bayesian inference. However, it is challenging to assess conditional density estimates and gain insight into modes of failure. While existing diagnostic tools can determine whether an approximated conditional density is compatible overall with a data sample, they lack a principled framework for identifying, locating, and interpreting the nature of statistically significant discrepancies over the entire feature space. In this paper, we present rigorous and easy-to-interpret diagnostics such as (i) the "Local Coverage Test" (LCT), which distinguishes an arbitrarily misspecified model from the true conditional density of the sample, and (ii) "Amortized Local P-P plots" (ALP) which can quickly provide interpretable graphical summaries of distributional differences at any location x in the feature space. Our validation procedures scale to high dimensions and can potentially adapt to any type of data at hand. We demonstrate the effectiveness of LCT and ALP through a simulated experiment and applications to prediction and parameter inference for image data.

We describe the design and use of linear hyperplanes to partition the feature space of a classifier. The objective of the partitioning is to minimize the average entropy of the class distribution in the final partitions. The hyperplanes are characterized by a vector /spl nu//sub n/ and scalar h/sub n/, which are computed with the objective of maximizing the mutual information associated with the partitioning. We show that the problem of designing the /spl nu//sub n/ can be simplified to an approximately equivalent problem, the solution of which, interestingly enough, turns out to be the linear discriminant of the data. We also describe a decision-tree based technique that partitions the feature space in a hierarchical fashion where each node in the decision tree represents a linear hyperplane that further partitions the feature space into two regions. The end result of the tree-growing process is that the entire feature space is partitioned into nonoverlapping regions where each region is bounded by a number of hyperplanes. As the criterion for the design of the hyperplanes is the minimization of the average class entropy of the regions, each region is characterized by the occurrence of only a small number of classes. The partitioning information provided by the decision-tree can then be used to eliminate a number of classes from being considered, and hence simplifies the job of the classifier. We show the application of the decision-tree as a preprocessor to a classifier in the speech recognition problem. Here the classes are modeled with mixtures of tens to hundreds of thousands of Gaussians, and the use of the partitioning information reduces the computation associated with the classification process by factors larger than 20 with negligible degradation in the word error rate (note, however, that the overall decoding process comprises other steps in addition to the classification-the computational speedup mentioned above does not affect the computation of these other steps). We also compare the performance of this quantization scheme with standard Gaussian clustering schemes and show that the decision-tree based quantization provides better performance in most cases.

Electrocardiogram (ECG) classification is a critical task for early detection of cardiac abnormalities.However, standard machine learning classifiers such as Logistic Regression (LR), K-Nearest Neighbors (KNN), Support Vector Machine (SVM), and Random Forest (RF) face challenges when handling high-dimensional data.These classifiers often struggle to model complex relationships among features, leading to reduced classification performance.Existing metaheuristic-based feature selection methods typically search across the entire feature space, which increases computational complexity and often fails to isolate only the most informative features.This paper introduces a novel hybrid feature selection approach based on a two-phase framework combining Topsoe Distance (TD) and Particle Swarm Optimization (PSO).In the first phase, TD is used to eliminate irrelevant and redundant features, thereby reducing the feature space from 4000 to 1600 and preserving only the most relevant and harmonized features.In the second phase, PSO is applied to the reduced space, enabling efficient optimization and feature subset selection.Experimental results using the MIT-BIH Arrhythmia dataset demonstrate that the proposed TD-PSO method significantly improves classification accuracy compared to using PSO alone.Specifically, LR accuracy improved from 87% to 95%, KNN from 92% to 97%, SVM from 92% to 96%, and RF from 93% to 99%.These results confirm that narrowing the search space before optimization allows classifiers to perform more effectively while reducing computational cost.The proposed TD-PSO framework provides a robust and scalable solution for high-dimensional ECG classification and holds strong potential for broader applications in biomedical data analysis.

There are two main paradigms in combining classifiers: classifier selection, where each classifier is considered as an expert in some local area of the feature space, and classifier fusion, where all classifiers are trained over the entire feature space and they are considered as competitive and complementary to each other. In this paper, we propose a new ensemble technique, Neuro-Ensemble, that follows the classifier fusion paradigm. The Neuro-Ensemble exploits the idea that different classifiers participating in the ensemble have varying degrees of expertise on learning different class labels and it optimizes the ensemble using a shallow Multi-Layer Perceptron (MLP) based meta-learner to capture the expertise of individual classifiers. Every neuron in the MLP represents a classifier that contributes with a vote and performs activation and state computations. This work is the first attempt to train a neural network for learning the expertise of each classifier in an ensemble and optimize the entire classification schema based on class-level expertise weights. We validated our Neuro-Ensemble on real-world datasets from the UCI repository. Our experimental results show the effectiveness and efficiency of our approach in comparison with individual baseline learners and ensemble techniques.

A novel feature extraction method that utilizes nonlinear mapping from the original data space to the feature space is presented in this dissertation. Feature extraction methods aim to find compact representations of data that are easy to classify. Measurements with similar values are grouped to same category, while those with differing values are deemed to be of separate categories. For most practical systems, the meaningful features of a pattern class lie in a low dimensional nonlinear constraint region (manifold) within the high dimensional data space. A learning algorithm to model this nonlinear region and to project patterns to this feature space is developed. Least squares estimation approach that utilizes interdependency between points in training patterns is used to form the nonlinear region. The proposed feature extraction strategy is employed to improve face recognition accuracy under varying illumination conditions and facial expressions. Though the face features show variations under these conditions, the features of one individual tend to cluster together and can be considered as a neighborhood. Low dimensional representations of face patterns in the feature space may lie in a nonlinear constraint region, which when modeled leads to efficient pattern classification. A feature space encompassing multiple pattern classes can be trained by modeling a separate constraint region for each pattern class and obtaining a mean constraint region by averaging all the individual regions. Unlike most other nonlinear techniques, the proposed method provides an easy intuitive way to place new points onto a nonlinear region in the feature space. The proposed feature extraction and classification method results in improved accuracy when compared to the classical linear representations. Face recognition accuracy is further improved by introducing the concepts of modularity, discriminant analysis and phase congruency into the proposed method. In the modular approach, feature components are extracted from different sub-modules of the images and concatenated to make a single vector to represent a face region. By doing this we are able to extract features that are more representative of the local features of the face. When projected onto an arbitrary line, samples from well formed clusters could produce a confused mixture of samples from all the classes leading to poor recognition. Discriminant analysis aims to find an optimal line orientation for which the data classes are well separated. Experiments performed on various databases to evaluate the performance of the proposed face recognition technique have shown improvement in recognition accuracy, especially under varying illumination conditions and facial expressions. This shows that the integration of multiple subspaces, each representing a part of a higher order nonlinear function, could represent a pattern with variability. Research work is progressing to investigate the effectiveness of subspace projection methodology for building manifolds with other nonlinear functions and to identify the optimum nonlinear function from an object classification perspective.

Multi-view representation learning is challenging because different views contain both the common structure and the complex view specific information. The traditional generative models may not be effective in such situation, since view-specific and common information cannot be well separated, which may cause problems for downstream vision tasks. In this paper, we introduce a multi-view generator model to solve the problem of multi-view generation and recognition in a unified framework. We propose a multi-view alternating back-propagation algorithm to learn multi-view generator networks by allowing them to share common latent factors. Our experiments show that the proposed method is effective for both image generation and recognition. Specifically, we first qualitatively demonstrate that our model can rotate and complete faces accurately. Then we show that our model can achieve state-of-art or competitive recognition performances through quantitative comparisons.

To create a pose-invariant face recognizer, one strategy is the view-based approach, which uses a set of real example views at different poses. But what if we only have one real view available, such as a scanned passport photo-can we still recognize faces under different poses? Given one real view at a known pose, it is still possible to use the view-based approach by exploiting prior knowledge of faces to generate virtual views, or views of the face as seen from different poses. To represent prior knowledge, we use 2D example views of prototype faces under different rotations. We develop example-based techniques for applying the rotation seen in the prototypes to essentially "rotate" the single real view which is available. Next, the combined set of one real and multiple virtual views is used as example views for a view-based, pose-invariant face recognizer. Oar experiments suggest that among the techniques for expressing prior knowledge of faces, 2D example-based approaches should be considered alongside the more standard 3D modeling techniques.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Multi-label learning has been successfully applied to solve instance multi-semantics problems. Moreover, the topology information of samples is often adopted in existing works to improve the prediction performance, in which the similarity of samples is usually calculated in the entire feature space. However, in real-world applications, each label is often determined by a subset of the original features, so when we focus on different labels, the similarity of two instances may be different. In this paper, we propose a multi-label learning method by exploiting the local similarity of samples. Specifically, the smoothness assumption is applied to assume that if the feature subset is similar between samples, the corresponding label should be similar. In addition, L1 regularization is also adopted to sparse the weight coefficients when constraining the output space of the instance. The experimental results on several data sets validate the effectiveness of the proposed method.

Representation-based classification have received much attention in the field of face recognition. Collaborative representation-based classification (CRC) has shown the robustness and high performance. In this paper, we proposed a new feature extraction method-based collaborative representation. Firstly, we get the coefficients of all face samples by collaborative representation. Then we define the inter-class reconstructive errors and intra-class reconstructive errors for each sample. After that, Fisher criterion is used to get the discriminative feature. At last, CRC is executed to get the identification results in the new feature space. Different from other feature extraction methods, the proposed method integrates the classification criterion into the feature extraction. So the feature space we get fits the classifier better. Experiment results on several face databases show that the proposed method is more effective than other state-of-the-art face recognition methods.

A recognition system based on fuzzy set theory and approximate reasoning that is capable of handling various imprecise input patterns and providing a natural decision system is described. The input feature is considered to be of either quantitative form, linguistic form, mixed form, or set form. The entire feature space is decomposed into overlapping subdomains depending on the geometric structure and the relative position of the pattern classes found in the training samples. Uncertainty (ambiguity) in the input statement is managed by providing/modifying membership values to a great extent. A relational matrix corresponding to the subdomains and the pattern classes is used to recognize the samples. The system uses L.A. Zadeh's (1977) compositional rule of inference and gives a natural (linguistic) multivalued output decision associated with a confidence factor denoting the degree of certainty of a decision. The effectiveness of the algorithm is demonstrated for some artificially generated patterns and for real-life speech data. >

Introducing clustering based population in Binary Gravitational Search Algorithm for Feature Selection

Datasets that are gathered from multiple sources are called multi-view datasets. These types of datasets represent different sets of feature attributes to form different views. Advanced computing and information technology have made it possible to collect and store a massive amount of data. As more features are added to each view, the data becomes sparse, and analysis suffers from the curse of dimensionality. Exploring and integrating diverse information from different views have been the focus of many approaches to improve clustering performance. The relationships among distinct views should not only be explored and analyze but should also consider the emerging high-dimensionality of each view to further improve the clustering performance. Recently, Yang and Benjamin proposed a clustering algorithm called “Feature Weighted Reduction PCM (FW-R-PCM) that calculates feature weights to identify relevant features and consequently, eliminates features that are irrelevant to reduce feature dimension from the entire feature space. In this paper, we propose a feature-weighted multi-view possibilistic c-means (FW-R-MVPCM) clustering algorithm, which extends FW-R-PCM to consider clustering high-dimensional multi-view datasets and perform feature reduction simultaneously. Our proposed FW-R-MVPCM can effectively improve the clustering performance using a weighting scheme that identifies, and selects relevant features from each view, and eliminate the irrelevant features, thus reducing the dimension of features in each view. Experiments on real datasets are performed to analyze the theoretical behavior of FW-R-MVPCM and to show its usefulness and effectiveness. Furthermore, FW-R-MVPCM is compared with FW-R-PCM, and other multi-view clustering algorithms such as WMCFS, SWVF, W-MV-PCM-L2, and WV-Co-FCM.

Incremental methods for structure learning of pairwise Markov random fields (MRFs), such as grafting, improve scalability by avoiding inference over the entire feature space in each optimization step. Instead, inference is performed over an incrementally grown active set of features. In this paper, we address key computational bottlenecks that current incremental techniques still suffer by introducing best-choice edge grafting, an incremental, structured method that activates edges as groups of features in a streaming setting. The method uses a reservoir of edges that satisfy an activation condition, approximating the search for the optimal edge to activate. It also reorganizes the search space using search-history and structure heuristics. Experiments show a significant speedup for structure learning and a controllable trade-off between the speed and quality of learning.