Hierarchical Multi-label Classification Problems: An LCS Approach

This paper proposes a hierarchical multi-label classification method based on SVM with composite kernel for solving gene function prediction. The hierarchical multi-label classification problem is resolved into a set of binary classification tasks. A composite kernel based SVM (ck-SVM) is introduced to deal with the binary classification tasks. In estimation procedure of ck-SVM, a supervised clustering with over-sampling strategy is introduced for solving imbalance dataset learning problem and improve classification performance. Experimental results on benchmark datasets demonstrate that the proposed method improves the classification performance efficiently.

An evaluation of global-model hierarchical classification algorithms for hierarchical classification problems with single path of labels

Classification is a common task in Machine Learning and Data Mining. Some classification problems are called hierarchical classification problems because they need to take into account a hierarchical taxonomy which establishes an order between involved classes. The protein's function prediction is considered a hierarchical classification problem because their functions are arranged in a hierarchical taxonomy of classes. This paper presents an algorithm for hierarchical classification using the jumping emerging patterns approach. Jumping emerging patterns have been used to flat classification and in this work we explore its adoption in a hierarchical classification scenario. The proposed algorithm was evaluated in eight real datasets, compared against two other recent hierarchical classification algorithms from the literature and also with flat k-nearest neighbor classification algorithm. Preliminary results showed that the proposed approach is an alternative for hierarchical classification, having as main advantages the simplicity and understandability with good accuracy.

The classification problems described in the Machine Learning literature usually relate to the classification of data in which each example is associated to a class belonging to a finite set of classes, all at the same level. However, there are classification issues, of a hierarchical nature, where the classes can be either subclasses or super classes of other classes. In many hierarchical problems, one or more examples may be associated with more than one class simultaneously. These problems are known as hierarchical multi-label classification (HMC) problems. In this work, the ML-KNN algorithm was used to predict hierarchical multi-label problems, in order to determine the number of classes that can be assigned to an example. Through the experiments performed on 10 protein function databases and the statistical analysis of the results, it can be shown that the adaptations performed in the ML-KNN algorithm brought significant performance improvements based on the hierarchical precision and recall metrics Hierarchical.

Hierarchical Classification is a very important classification task for arranging data in a hierarchical structure. Hierarchical arrangement of data is one of the best methods to achieve better understanding of complex data. In this paper, we propose the HMAC method to perform Hierarchical Multi-label Associative Classification. This method uses multiple and negative rules to predict class-set and to filter exceptional cases in order to improve both accuracy and explanatory ability of the resulting classifier. Redundant rule pruning method for negative rules and rule ranking method for hierarchical associative classification are developed. Moreover, we propose a rule evaluation measure, Sim, that tread-off between F-measure and Jaccard's coefficient to encode hierarchical structure meaning of actual and predicted node collections. We show that Sim is robust and simple rule evaluation measure. Experimental results show that HMAC improves accuracy and explanatory ability of hierarchical classifiers compared with various rule ranking and pruning techniques.

In this paper we address the task of hierarchical bird species identification from audio recordings. We evaluate three types of approaches to deal with hierarchical classification problems: the flat classification approach, the local-model per parent node classifier approach and the global-model hierarchical-classification approach. For the flat and local-model classification approach we employ the classic Naive Bayes algorithm. For the global-model approach we use the Global Model Naive Bayes (GMNB) algorithm. As in the classical Naive Bayes, the algorithm computes prior probabilities and likelihoods, but these computations take into account the hierarchical classification scenario: it assumes that any example which belongs to a given class will also belong to all its ancestor classes. In the current application, the employed class hierarchy is the standard scientific taxonomy of birds used in Biology. In order to deal with the bird songs we obtain features by computing several acoustic quantities from intervals of the audio signal. We conduct three experiments in order to compare the three different approaches to the hierarchical bird species identification problem. Our experimental results show that the use of the GMNB hierarchical classification algorithm outperforms both the flat and local-model approaches (Using the Hierarchical F-measure metric), hence the use of a global-model approach (such as the GMNB) can be a feasible way to improve the classification performance for problems with a large number of classes.

Recent advances in science and technology have made possible the data growth in quantity and availability. Along with this explosion of generated information, there is a need to analyze data to discover new and useful knowledge. Thus, areas for extracting knowledge and useful information in large datasets have become great opportunities for the advancement of research, such as Machine Learning (ML) and Data Mining (DM). However, there are some limitations that may reduce the accuracy of some traditional algorithms of these areas, for example the imbalance of classes samples in a dataset. To mitigate this drawback, some solutions have been the target of research in recent years, such as the development of techniques for artificial balancing data, algorithm modification and new approaches for imbalanced data. An area little explored in the data imbalance vision are the problems of hierarchical classification, in which the classes are organized into hierarchies, commonly in the form of tree or DAG (Direct Acyclic Graph). The goal of this work aims at investigating the limitations and approaches to minimize the effects of imbalanced data with hierarchical classification problems. The experimental results show the need to take into account the features of hierarchical classes when deciding the application of techniques for imbalanced data in hierarchical classification.

Proteins are the main building blocks of the cell, and perform almost all the functions related to cell activity. Despite the recent advances in Molecular Biology, the function of a large amount of proteins is still unknown. The use of algorithms able to induce classification models is a promising approach for the functional prediction of proteins, whose classes are usually organized hierarchically. Among the machine learning techniques that have been used in hierarchical classification problems, one may highlight the Decision Trees. This paper describes the main characteristics of hierarchical classification models for Bioinformatics problems and applies three hierarchical methods based on the use of Decision Trees to protein functional classification datasets.KeywordsGene OntologyLeaf NodeFourth LevelMain Building BlockInterpro EntryThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Learning classifier systems (LCSs) have been successfully adapted to real-world domains with the claim of human-readable rule populations. However, due to the inherent rich characteristic of the employed representation, it is possible to represent the underlying patterns in multiple (polymorphic) ways, which obscures the most informative patterns. A novel rule reduction algorithm is proposed based on ensembles of multiple trained LCSs populations in a hierarchical learning architecture to reduce the local diversity and global polymorphism. The primary aim of this project is to interrogate the hidden patterns in LCSs’ trained population rather than improve the predictive power on test sets. This enables successful visualization of the importance of features in data groups (niches) that can contain heterogeneous patterns, i.e. even if different patterns result in the same class the importance of features can be found.

The performance of emotive text classification using affective hierarchical schemes (e.g., WordNet-Affect) is often evaluated using the same traditional measures used to evaluate the performance of when a finite set of isolated classes are used. However, applying such measures means the full characteristics and structure of the emotive hierarchical scheme are not considered. Thus, the overall performance of emotive text classification using emotion hierarchical schemes is often inaccurately reported and may lead to ineffective information retrieval and decision making. This paper provides a comparative investigation into how methods used in hierarchical classification problems in other domains, which extend traditional evaluation metrics to consider the characteristics of the hierarchical classification scheme, can be applied and subsequently improve the classification of emotive texts. This study investigates the classification performance of three widely used classifiers, Naive Bayes, J48 Decision Tree, and SVM, following the application of the aforementioned methods. The results demonstrated that all the methods improved the emotion classification. However, the most notable improvement was recorded when a depth-based method was applied to both the testing and validation data, where the precision, recall, and F1-score were significantly improved by around 70 percentage points for each classifier.

Deep hiearchical multi-label classification applied to chest X-ray abnormality taxonomies.

In this paper, we present a hierarchical multi-label classification system for visual concepts detection and image annotation. Hierarchical multi-label classification (HMLC) is a variant of classification where an instance may belong to multiple classes at the same time and these classes/labels are organized in a hierarchy. The system is composed of two parts: feature extraction and classification/annotation. The feature extraction part provides global and local descriptions of the images. These descriptions are then used to learn a classifier and to annotate an image with the corresponding concepts. To this end, we use predictive clustering trees (PCTs), which are able to classify target concepts that are organized in a hierarchy. Our approach to HMLC exploits the annotation hierarchy by building a single predictive clustering tree that can simultaneously predict all of the labels used to annotate an image. Moreover, we constructed ensembles (random forests) of PCTs, to improve the predictive performance. We tested our system on the image database from the ImageCLEF@ICPR 2010 photo annotation task. The extensive experiments conducted on the benchmark database show that our system has very high predictive performance and can be easily scaled to large number of visual concepts and large amounts of data.

Quality control of crowdsourced classification using hierarchical class structures

In recent years, multi-label classification has attracted a significant body of research, motivated by real-life applications such as text classification and medical diagnoses. However, rule-based methods, and especially Learning Classifier Systems (LCS), for tackling such problems have only been sparsely studied. This is the motivation behind our current work that introduces a generalized multi-label rule format and uses it as a guide for further adapting the general Michigan-style LCS framework. The resulting LCS algorithm is thoroughly evaluated and found competitive to other state-of-the-art multi-label classification methods.Keywordsmulti-label classificationlearning classifier systemsgenetics-based machine learningclassificationevolutionary computation

Assigning the subject heading is a significant work of the librarian. Subject heading represents the overall contents of the books which is a great way to easily find the books related directly to the user’s needs. But a book can be related to more than one subject and Some librarians are not experts in every discipline. They take more time on this process. In this paper, our research has a small dataset, so the multi-label classification technique is unsuitable for this work. Therefore, we propose the hierarchical multi-label classification to help them assign subject heading to the book from a title and a table of contents. We also compare the performance of three techniques: Support Vector Machine (SVM), Random Forest (RF), and K-Nearest Neighbors (KNN), to select the best classification technique. According to the results, Random Forest technique is the best one for the first hierarchy classifiers with the highest F1 of 0.908. The second hierarchy classifiers that uses Random Forest technique attains the F1 score of 0.871.