- Research Article
86
- 10.1016/j.eswa.2021.115224
VMFS: A VIKOR-based multi-target feature selection
- May 21, 2021
- Expert Systems with Applications
- Amin Hashemi + 2 more +2
VMFS: A VIKOR-based multi-target feature selection
International audience
VMFS: A VIKOR-based multi-target feature selection
VMFS: A VIKOR-based multi-target feature selection
Towards Meta-Learning for Multi-Target Regression Problems
Several multi-target regression methods were developed in the last years aiming at improving predictive performance by exploring inter-target correlation within the problem. However, none of these methods outperforms the others for all problems. This motivates the development of automatic approaches to recommend the most suitable multi-target regression method. In this paper, we propose a meta-learning system to recommend the best predictive method for a given multi-target regression problem. We performed experiments with a meta-dataset generated by a total of 648 synthetic datasets. These datasets were created to explore distinct inter-targets characteristics toward recommending the most promising method. In experiments, we evaluated four different algorithms with different biases as meta-learners. Our meta-dataset is composed of 58 meta-features, based on: statistical information, correlation characteristics, linear landmarking, from the distribution and smoothness of the data, and has four different meta-labels. Results showed that induced meta-models were able to recommend the best method for different base level datasets with a balanced accuracy superior to 70% using a Random Forest meta-model, which statistically outperformed the meta-learning baselines.
Read moreRotation Forest for multi-target regression
The prediction of multiple numeric outputs at the same time is called multi-target regression (MTR), and it has gained attention during the last decades. This task is a challenging research topic in supervised learning because it poses additional difficulties to traditional single-target regression (STR), and many real-world problems involve the prediction of multiple targets at once. One of the most successful approaches to deal with MTR, although not the only one, consists in transforming the problem in several STR problems, whose outputs will be combined building up the MTR output. In this paper, the Rotation Forest ensemble method, previously proposed for single-label classification and single-target regression, is adapted to MTR tasks and tested with several regressors and data sets. Our proposal rotates the input space in an efficient and novel fashion, avoiding extra rotations forced by MTR problem decomposition. Four approaches for MTR are used: single-target (ST), stacked-single target (SST), Ensembles of Regressor Chains (ERC), and Multi-target Regression via Quantization (MRQ). For assessing the benefits of the proposal, a thorough experimentation with 28 MTR data sets and statistical tests are used, concluding that Rotation Forest, adapted by means of these approaches, outperforms other popular ensembles, such as Bagging and Random Forest.
Read moreMulti-target regression from high-speed data streams with adaptive model rules
Many real life prediction problems involve predicting a structured output. Multi-target regression is an instance of structured output prediction whose task is to predict for multiple target variables. Structured output algorithms are usually computationally and memory demanding, hence are not suited for dealing with massive amounts of data. Most of these algorithms can be categorized as local or global methods. Local methods produce individual models for each output component and combine them to produce the structured prediction. Global methods adapt traditional learning algorithms to predict the output structure as a whole. We propose the first rule-based algorithm for solving multi-target regression problems from data streams. The algorithm builds on the adaptive model rules framework. In contrast to the majority of the structured output predictors, this particular algorithm does not fall into the local and global categories. Instead, each rule specializes on related subsets of the output attributes. To evaluate the performance of the proposed algorithm, two other rule-based algorithms were developed, one using the local strategy and the other using the global strategy. These methods were compared considering their prediction error, memory usage, computational time, and model complexity. Experimental results on synthetic and real data show that the local-strategy algorithm usually obtains the lowest error. However, the proposed and the global-strategy algorithms use much less memory and run significantly much faster at the cost of a slightly increase in the error, which make them very attractive when computation resources are an important factor. Also, the models produced by the latter approaches are much easier to understand since considerably less rules are produced.
Read moreFeature ranking for semi-supervised learning
The data used for analysis are becoming increasingly complex along several directions: high dimensionality, number of examples and availability of labels for the examples. This poses a variety of challenges for the existing machine learning methods, related to analyzing datasets with a large number of examples that are described in a high-dimensional space, where not all examples have labels provided. For example, when investigating the toxicity of chemical compounds, there are many compounds available that can be described with information-rich high-dimensional representations, but not all of the compounds have information on their toxicity. To address these challenges, we propose methods for semi-supervised learning (SSL) of feature rankings. The feature rankings are learned in the context of classification and regression, as well as in the context of structured output prediction (multi-label classification, MLC, hierarchical multi-label classification, HMLC and multi-target regression, MTR) tasks. This is the first work that treats the task of feature ranking uniformly across various tasks of semi-supervised structured output prediction. To the best of our knowledge, it is also the first work on SSL of feature rankings for the tasks of HMLC and MTR. More specifically, we propose two approaches—based on predictive clustering tree ensembles and the Relief family of algorithms—and evaluate their performance across 38 benchmark datasets. The extensive evaluation reveals that rankings based on Random Forest ensembles perform the best for classification tasks (incl. MLC and HMLC tasks) and are the fastest for all tasks, while ensembles based on extremely randomized trees work best for the regression tasks. Semi-supervised feature rankings outperform their supervised counterparts across the majority of datasets for all of the different tasks, showing the benefit of using unlabeled in addition to labeled data.
Read moreAn empirical comparison on multi-target regression learning
Multi-target regression is concerned with the simultaneous prediction of multiple continuous target variables based on the same set of input variables. It has received relatively small attention from the Machine Learning community. However, multi-target regression exists in many real-world applications. In this paper we conduct extensive experiments to investigate the performance of three representative multi-target regression learning algorithms (i.e., multi-target stacking (MTS), random linear target combination (RLTC), and multi-objective random forest (MORF)), comparing the baseline single-target learning. Our experimental results show that all three multi-target regression learning algorithms do improve the performance of the single-target learning. Among them, MTS performs the best, followed by RLTC, followed by MORF. However, the single-target learning sometimes still performs very well, even the best. This analysis sheds the light on multi-target regression learning and indicates that the single-target learning is a competitive baseline for multi-target regression learning on multi-target domains.
Read moreSemi-supervised trees for multi-target regression
Semi-supervised trees for multi-target regression
Consensus unsupervised feature ranking from multiple views
Consensus unsupervised feature ranking from multiple views
From Document to Image: Learning a Scalable Ranking Model for Content Based Image Retrieval
With the recent advancement of web search ranking framework, a.k.a. learning to rank, it is questionable whether it can be still applicable to the large-scale content based image retrieval settings. Moreover, given the complex structure of image representation, it is also challenging how to design visual ranking features that not only scale up well, but also model various visual modalities and the spatial distributions of local features. In this paper, we answer the above two questions by investigating the performance of learning to rank for the large-scale content based image retrieval problem, with some scalable visual based ranking features proposed to improve the performance. Specifically, we firstly adopt several well performed ad-hoc ranking models to generate the Bag-of-Visual-Words based ranking features. Additionally, to preserve the spatial information of image local descriptors, we split images into blocks from coarse to fine, and extract ranking features hierarchically with a spatial pyramid manner. Finally, image global features are also quantized via LSH and concatenated with the existing ranking features all together. Experimental results on both Oxford and Image Net databases demonstrate the effectiveness and efficiency of the proposed ranking model, as well as the complementarity of each ranking features.
Read moreLearning Local and Global Features for Optimized Multi-Label Text Classification
In multi-label text classification, the central aim is to associate an array of descriptive labels for a better understanding of the text. There are three main challenges in doing multi-label text classification (i) a large number of text (input) features, (ii) the underlying implicit relationship between input features and output labels, and (iii) an implicit inter-label dependency. In traditional approaches to multi-label classification, these problems are not being addressed collectively. A feature selection strategy that inherently uses local features to discriminate a class and similarly global features that can distinctly separate classes can be very effective for multi-label classification. In this research, we perform a feature selection and ranking strategy based on local and global features. A Naïve Bayes classifier is being used using a combination of these two -feature sets, it is compared with the baseline implemented with the term frequency-inverse document frequency (TF-IDF). A series of experiments have been carried out on standard multi-label text datasets, using evaluation metrics like Hamming loss, Subset Accuracy and Micro/Macro F1 scores, and encouraging results are obtained.
Read moreSaliency detection method for probabilistic hypergraph ranking based on local features
The saliency detection algorithm achieves object detection by acquiring salient regions in an image. We propose a saliency detection algorithm based on probabilistic hypergraph ranking of local features. Firstly, a probabilistic hypergraph structure based on spatial location is constructed based on the connectivity of background nodes, and the background saliency map is built using the ranking algorithm; Secondly, according to the clustering of foreground nodes, a probabilistic hypergraph based on feature clustering is constructed, and the foreground saliency map is realized using the ranking; Finally, the background and foreground saliency map are fused to obtain the saliency detection results. The simulation experiment results show that the detection accuracy of the algorithm we proposed is higher.
Read moreMultimodal multitask deep learning model for Alzheimer’s disease progression detection based on time series data
Multimodal multitask deep learning model for Alzheimer’s disease progression detection based on time series data
Investigating Feature Ranking Methods for Sub-Band and Relative Power Features in Motor Imagery Task Classification
Interpreting the brain commands is now easier using brain-computer interface (BCI) technologies. Motor imagery (MI) signal detection is one of the BCI applications, where the movements of the hand and feet can be recognized via brain commands that can be further used to handle emergency situations. Design of BCI techniques encountered challenges of BCI illiteracy, poor signal to noise ratio, intersubject variability, complexity, and performance. The automated models designed for emergency should have lesser complexity and higher performance. To deal with the challenges related to the complexity performance tradeoff, the frequency features of brain signal are utilized in this study. Feature matrix is created from the power of brain frequencies, and newly proposed relative power features are used. Analysis of the relative power of alpha sub-band to beta, gamma, and theta sub-band has been done. These proposed relative features are evaluated with the help of different classifiers. For motor imagery classification, the proposed approach resulted in a maximum accuracy of 93.51% compared to other existing approaches. To check the significance of newly added features, feature ranking approaches, namely, mutual information, chi-square, and correlation, are used. The ranking of features shows that the relative power features are significant for MI task classification. The chi-square provides the best tradeoff between accuracy and feature space. We found that the addition of relative power features improves the overall performance. The proposed models could also provide quick response having reduced complexity.
Read moreClassifiers fusion for improved vessel recognition with application in quantification of generalized arteriolar narrowing
This paper attempts to estimate diagnostically relevant measure, i.e., Arteriovenous Ratio with an improved retinal vessel classification using feature ranking strategies and multiple classifiers decision-combination scheme. The features exploited for retinal vessel characterization are based on statistical measures of histogram, different filter responses of images and local gradient information. The feature selection process is based on two feature ranking approaches (Pearson Correlation Coefficient technique and Relief-F method) to rank the features followed by use of maximum classification accuracy of three supervised classifiers (k-Nearest Neighbor, Support Vector Machine and Naïve Bayes) as a threshold for feature subset selection. Retinal vessels are labeled using the selected feature subset and proposed hybrid classification scheme, i.e., decision fusion of multiple classifiers. The comparative analysis shows an increase in vessel classification accuracy as well as Arteriovenous Ratio calculation performance. The system is tested on three databases, a local dataset of 44 images and two publically available databases, INSPIRE-AVR containing 40 images and VICAVR containing 58 images. The local database also contains images with pathologically diseased structures. The performance of the proposed system is assessed by comparing the experimental results with the gold standard estimations as well as with the results of previous methodologies. Overall, an accuracy of 90.45%, 93.90% and 87.82% is achieved in retinal blood vessel separation with 0.0565, 0.0650 and 0.0849 mean error in Arteriovenous Ratio calculation for Local, INSPIRE-AVR and VICAVR dataset, respectively.
Read moreFuzzy Jaccard Index: A robust comparison of ordered lists
Fuzzy Jaccard Index: A robust comparison of ordered lists