Discriminating Joint Feature Analysis for Multimedia Data Understanding

A robust graph-based semi-supervised sparse feature selection method

The explosive growth of digital images requires effective methods to manage these images. Among various existing methods, automatic image annotation has proved to be an important technique for image management tasks, e.g., image retrieval over large-scale image databases. Automatic image annotation has been widely studied during recent years and a considerable number of approaches have been proposed. However, the performance of these methods is yet to be satisfactory, thus demanding more effort on research of image annotation. In this paper, we propose a novel semi supervised framework built upon feature selection for automatic image annotation. Our method aims to jointly select the most relevant features from all the data points by using a sparsity-based model and exploiting both labeled and unlabeled data to learn the manifold structure. Our framework is able to simultaneously learn a robust classifier for image annotation by selecting the discriminating features related to the semantic concepts. To solve the objective function of our framework, we propose an efficient iterative algorithm. Extensive experiments are performed on different real-world image datasets with the results demonstrating the promising performance of our framework for automatic image annotation.

This study discusses the problem of feature selection as one of the most fundamental problems in the field of the machine learning. Two novel approaches for feature selection in order to select a subset with relevant features are proposed. These approaches can be considered as a direct extension of the ensemble feature selection approach. The first one deals with identifying relevant features by using a single feature selection method. While, the second one uses different feature selection methods in order to identify more correctly the relevant features. An illustration shows the effectiveness of the proposed methods on artificial databases where we have a priori the informations about the relevant features.KeywordsFeature SelectionRelevant FeatureFeature SubsetFeature Selection MethodNeural Information Processing SystemThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Unsupervised feature selection has attracted more and more attention due to the rapid growth of the large amount of unlabelled and high-dimensional data. The performance of traditional spectral-based unsupervised methods always depends on the quality of constructed similarity matrix. However, real world data always contain a large number of noise samples and features that make the similarity matrix created by original data cannot be fully relied. We propose an unsupervised feature selection method which conducts feature selection and local structure learning simultaneously. Moreover, we add an important constraint on the similarity matrix to allow it to capture more accurate information of the data structure. To perform feature selection, orthogonal constraint and $\ell _{2,p}$ l 2 , p -norm are adopted on the projection matrix. An efficient and simple algorithm is derived to tackle the problem. We conduct comprehensive experiments on various benchmark data sets, including handwritten digit, face image, and biomedical data, to validate the effectiveness of the proposed approach.

High-feature dimensions is one of the obvious challenges facing large-scale hierarchical classification problem. One promising way to overcome such problem is reducing the features size by removing irrelevant, noise and redundant features. Filter-based feature selection methods are commonly used to construct smaller subset of most relevant features in either adaptive or global approach at each non leaf node within the hierarchy. Both the Feature selection approaches do not only reduce time and space complexity but also improve classification accuracy of the learning models. In this study, the effectiveness of the main approaches for constructing relevant features subset, that is adaptive and global approaches are analyzed using the two filter-based methods suitable for text classification. Experiment evaluation conducted on 20NG, IPC, and DMOZ-small datasets shows that implementing feature selection adaptively has more capability of selecting most relevant features, thus, achieving the best classification accuracy. Moreover, further evaluation shown integrating feature selection with LSHC records improve performance when compared to without feature selection.

<p>Classification problems often have a large number of features, but not all of them are useful for classification. Irrelevant and redundant features may even reduce the classification accuracy. Feature selection is a process of selecting a subset of relevant features, which can decrease the dimensionality, shorten the running time, and/or improve the classification accuracy. There are two types of feature selection approaches, i.e. wrapper and filter approaches. Their main difference is that wrappers use a classification algorithm to evaluate the goodness of the features during the feature selection process while filters are independent of any classification algorithm. Feature selection is a difficult task because of feature interactions and the large search space. Existing feature selection methods suffer from different problems, such as stagnation in local optima and high computational cost. Evolutionary computation (EC) techniques are well-known global search algorithms. Particle swarm optimisation (PSO) is an EC technique that is computationally less expensive and can converge faster than other methods. PSO has been successfully applied to many areas, but its potential for feature selection has not been fully investigated. The overall goal of this thesis is to investigate and improve the capability of PSO for feature selection to select a smaller number of features and achieve similar or better classification performance than using all features. This thesis investigates the use of PSO for both wrapper and filter, and for both single objective and multi-objective feature selection, and also investigates the differences between wrappers and filters. This thesis proposes a new PSO based wrapper, single objective feature selection approach by developing new initialisation and updating mechanisms. The results show that by considering the number of features in the initialisation and updating procedures, the new algorithm can improve the classification performance, reduce the number of features and decrease computational time. This thesis develops the first PSO based wrapper multi-objective feature selection approach, which aims to maximise the classification accuracy and simultaneously minimise the number of features. The results show that the proposed multi-objective algorithm can obtain more and better feature subsets than single objective algorithms, and outperform other well-known EC based multi-objective feature selection algorithms. This thesis develops a filter, single objective feature selection approach based on PSO and information theory. Two measures are proposed to evaluate the relevance of the selected features based on each pair of features and a group of features, respectively. The results show that PSO and information based algorithms can successfully address feature selection tasks. The group based method achieves higher classification accuracies, but the pair based method is faster and selects smaller feature subsets. This thesis proposes the first PSO based multi-objective filter feature selection approach using information based measures. This work is also the first work using other two well-known multi-objective EC algorithms in filter feature selection, which are also used to compare the performance of the PSO based approach. The results show that the PSO based multiobjective filter approach can successfully address feature selection problems, outperform single objective filter algorithms and achieve better classification performance than other multi-objective algorithms. This thesis investigates the difference between wrapper and filter approaches in terms of the classification performance and computational time, and also examines the generality of wrappers. The results show that wrappers generally achieve better or similar classification performance than filters, but do not always need longer computational time than filters. The results also show that wrappers built with simple classification algorithms can be general to other classification algorithms.</p>

Big multimedia data is heterogeneous in essence, that is, the data may be a mixture of video, audio, text, and images. This is due to the prevalence of novel applications in recent years, such as social media, video sharing, and location based services (LBS), etc. In many multimedia applications, for example, video/image tagging and multimedia recommendation, text classification techniques have been used extensively to facilitate multimedia data processing. In this paper, we give a comprehensive review on feature selection techniques for text classification. We begin by introducing some popular representation schemes for documents, and similarity measures used in text classification. Then, we review the most popular text classifiers, including Nearest Neighbor (NN) method, Naive Bayes (NB), Support Vector Machine (SVM), Decision Tree (DT), and Neural Networks. Next, we survey four feature selection models, namely the filter, wrapper, embedded and hybrid, discussing pros and cons of the state-of-the-art feature selection approaches. Finally, we conclude the paper and give a brief introduction to some interesting feature selection work that does not belong to the four models.

Semi-supervised multi-label feature selection with adaptive structure learning and manifold learning

Graph self-representation method for unsupervised feature selection

Feature selection is the method of reducing the size of data without degrading their accuracy. In this study, we propose a novel feature selection approach, based on extreme learning machines (ELMs) and the coefficient of variation (CV). In the proposed approach, the most relevant features are identified by ranking each feature with the coefficient obtained through ELM divided by CV. The achieved accuracies and computational costs, obtained with the use of features selected via the proposed approach in 9 classification and 26 regression benchmark data sets, were compared to those obtained with all features, as well as those obtained with the features selected by a wrapper and a filtering method. The achieved accuracy values obtained with the proposed approach were generally higher than when using all features. Furthermore, high feature reduction ratios were obtained with the proposed approach, including the achieved feature reduction ratios in epilepsy, liver, EMG, shuttle, and abalone. Stock data sets were 90.48%, 90%, 70.59%, 66.67%, 75%, and 77.78%, respectively. This approach is an extremely fast process that is independent of the employed machine-learning methods.

The development of technology generates huge amounts of non-textual information, such as images. An efficient image annotation and retrieval system is highly desired. Clustering algorithms make it possible to represent visual features of images with finite symbols. Based on this, many statistical models, which analyze correspondence between visual features and words and discover hidden semantics, have been published. These models improve the annotation and retrieval of large image databases. However, image data usually have a large number of dimensions. Traditional clustering algorithms assign equal weights to these dimensions, and become confounded in the process of dealing with these dimensions. In this paper, we propose weighted feature selection algorithm as a solution to this problem. For a given cluster, we determine relevant features based on histogram analysis and assign greater weight to relevant features as compared to less relevant features. We have implemented various different models to link visual tokens with keywords based on the clustering results of K-means algorithm with weighted feature selection and without feature selection, and evaluated performance using precision, recall and correspondence accuracy using benchmark dataset. The results show that weighted feature selection is better than traditional ones for automatic image annotation and retrieval.

Machine learning models have been successfully employed in the diagnosis of Schizophrenia disease. The impact of classification models and the feature selection techniques on the diagnosis of Schizophrenia have not been evaluated. Here, we sought to access the performance of classification models along with different feature selection approaches on the structural magnetic resonance imaging data. The data consist of 72 subjects with Schizophrenia and 74 healthy control subjects. We evaluated different classification algorithms based on support vector machine (SVM), random forest, kernel ridge regression and randomized neural networks. Moreover, we evaluated T-Test, Receiver Operator Characteristics (ROC), Wilcoxon, entropy, Bhattacharyya, Minimum Redundancy Maximum Relevance (MRMR) and Neighbourhood Component Analysis (NCA) as the feature selection techniques. Based on the evaluation, SVM based models with Gaussian kernel proved better compared to other classification models and Wilcoxon feature selection emerged as the best feature selection approach. Moreover, in terms of data modality the performance on integration of the grey matter and white matter proved better compared to the performance on the grey and white matter individually. Our evaluation showed that classification algorithms along with the feature selection approaches impact the diagnosis of Schizophrenia disease. This indicates that proper selection of the features and the classification models can improve the diagnosis of Schizophrenia.

A new hybrid filter–wrapper feature selection method for clustering based on ranking

The growth of Big Data has resulted in an overwhelming increase in the volume of data available, including the number of features. Feature selection, the process of selecting relevant features and discarding irrelevant ones, has been successfully used to reduce the dimensionality of datasets. However, with numerous feature selection approaches in the literature, determining the best strategy for a specific problem is not straightforward. In this study, we compare the performance of various feature selection approaches to a random selection to identify the most effective strategy for a given type of problem. We use a large number of datasets to cover a broad range of real-world challenges. We evaluate the performance of seven popular feature selection approaches and five classifiers. Our findings show that feature selection is a valuable tool in machine learning and that correlation-based feature selection is the most effective strategy regardless of the scenario. Additionally, we found that using improper thresholds with ranker approaches produces results as poor as randomly selecting a subset of features.

Processing huge amounts of network data is one of the largest challenges for network-based intrusion detection system (IDS). Usually these data contain lots of irrelevant or redundant features. To improve the efficiency of IDS, relevant features are necessary to be extracted from original data via feature selection approaches. In this paper, an effective feature selection approach based on Bayesian Network classifier is proposed. And with the same intrusion detection benchmark dataset (NSL-KDD), the performance of the proposed approach is evaluated and compared with other commonly used feature selection methods. It is shown by empirical results that features selected by our approach have decreased the time to detect attacks and increased the classification accuracy as well as the true positive rates significantly.