Hierarchical Multi-Label Classification of Library Subject Headings

Machine learning involves the task of training systems to be able to make decisions without being explicitly programmed. Important among machine learning tasks is classification involving the process of training machines to make predictions from predefined labels. Classification is broadly categorized into three distinct groups: single-label (SL), multi-class, and multi-label (ML) classification. This research work presents an application of a multi-label classification (MLC) technique in automating Quranic verses labeling. MLC has been gaining attention in recent years. This is due to the increasing amount of works based on real-world classification problems of multi-label data. In traditional classification problems, patterns are associated with a single-label from a set of disjoint labels. However, in MLC, an instance of data is associated with a set of labels. In this paper, three standard <em>MLC</em> methods: <span>binary relevance (BR), classifier chain (CC), and label powerset (LP) algorithms are implemented with four baseline classifiers: support vector machine (SVM), naïve Bayes (NB), k-nearest neighbors (k-NN), and J48. The research methodology adopts the multi-label problem transformation (PT) approach. The results are validated using six conventional performance metrics. These include: hamming loss, accuracy, one error, micro-F1, macro-F1, and avg. precision. From the results, the classifiers effectively achieved above 70% accuracy mark. Overall, SVM achieved the best results with CC and LP algorithms.</span>

This paper proposes a hierarchical multi-label classification method based on SVM with composite kernel for solving gene function prediction. The hierarchical multi-label classification problem is resolved into a set of binary classification tasks. A composite kernel based SVM (ck-SVM) is introduced to deal with the binary classification tasks. In estimation procedure of ck-SVM, a supervised clustering with over-sampling strategy is introduced for solving imbalance dataset learning problem and improve classification performance. Experimental results on benchmark datasets demonstrate that the proposed method improves the classification performance efficiently.

In this paper, we present a hierarchical multi-label classification system for visual concepts detection and image annotation. Hierarchical multi-label classification (HMLC) is a variant of classification where an instance may belong to multiple classes at the same time and these classes/labels are organized in a hierarchy. The system is composed of two parts: feature extraction and classification/annotation. The feature extraction part provides global and local descriptions of the images. These descriptions are then used to learn a classifier and to annotate an image with the corresponding concepts. To this end, we use predictive clustering trees (PCTs), which are able to classify target concepts that are organized in a hierarchy. Our approach to HMLC exploits the annotation hierarchy by building a single predictive clustering tree that can simultaneously predict all of the labels used to annotate an image. Moreover, we constructed ensembles (random forests) of PCTs, to improve the predictive performance. We tested our system on the image database from the ImageCLEF@ICPR 2010 photo annotation task. The extensive experiments conducted on the benchmark database show that our system has very high predictive performance and can be easily scaled to large number of visual concepts and large amounts of data.

Breast cancer is the most common type of cancer in Egyptian women. According to the International Agency for Research on Cancer, the causes of breast cancer are not yet fully known but some of the main risk factors have been identified and taken into consideration in researches performed on patients with high risk of breast cancer. In this paper various classification techniques are used to classify whether breast cancer is recurrent or non-recurrent for a number of patients. Classification techniques used are K-Nearest Neighbor (KNN), Decision Trees (DT), Naive Bayes (NB), Support Vector Machines (SVM) and ensemble techniques Bagging, Voting and Random Forest (RF). The dataset is taken from the University of California Irvine (UCI) machine learning repository and experiments are conducted with Waikato Environment for Knowledge Analysis (WEKA) data mining tool. The research conducted goes through two phases, in the first phase the Random Forest classifier produced the best results with (84.3%) accuracy and the second phase, voting ensemble classifier produced the best results of 89.9% accuracy. The system model show an improvement in the overall accuracy compared to other researches done on the same dataset.

Background/Objectives: Recent studies emphasized on using ensemble models over single ones to solve credit scoring problems. The objective of this study is to build a heterogeneous ensemble classifier model with an improved classification accuracy. Methods: This study focuses on developing a heterogeneous ensemble classifier using Logistic Regression, K-nearest neighbor, Decision tree, Random Forest, Naïve Base and Support vector machine as base classifiers and Random Forest, Logistic Regression and Support vector machine as meta-classifiers. The proposed model is built using these six base classifiers for ensemble aggregation. A feature selection algorithm based on the random forest technique is used for selecting the best features. A stacking and voting method are used for building ensemble model. Findings: The ensemble classifier gives superior predictive performance than single classifiers SVM, DT, RF, NB, KNN and LR with an accuracy of 91.56% for Australian dataset and 84.35% for German dataset. Novelty: The proposed model uses stacking and majority voting method for ensemble classification. Initially, stacking is applied to the base classifiers. This is done in two levels. First the training dataset is split into 10 folds for cross validation. The output of each classifier is taken, and the dataset is updated with the meta-features. In the second level, three meta-classifiers (MC), namely LR, SVM and RF are used. Majority voting is applied to the output of these meta-classifiers for the prediction. Keywords: Credit scoring; ensemble model; SVM; DT; RF; NB; KNN; LR

Deep hiearchical multi-label classification applied to chest X-ray abnormality taxonomies.

Hierarchical multi-label text classification (HMTC) has become rather challenging when it requires handling large sets of closely related categories. We present a novel software that provides a hybrid embedding-based text representation for HMTC, shortened as HE-HMTC. It made full use of categories’ structure and their labels semantics to enrich the text representation, and therefore attempted to improve the classification performance of the text. In addition, HE-HMTC facilitates improving the accuracy of HMTC tasks. Moreover, our HE-HMTC can easily be generalized in other hierarchical classification tasks and achieve superior performance.

Fault detection and classification of an electrical equipment is a significant subject concerning the continuity of efficient working and necessary tasks. The heat concept creates a stimulating effect in case of failure among the electrical equipment. For this reason, thermal camera images can be functional and are used to detect the malfunctions. In this paper, thermal camera images are utilized to detect 11 different conditions of induction motors that are 8 different short-circuit faults of stator windings, rotor failure, cooling fan failure, and no-load. First-order statistics (FOS) are considered to obtain the discriminative information among the thermal images. The classification unit of model is specified examining five efficient algorithms that are neural networks (NN), k-nearest neighbors (k-NN), random forest (RF), logistic regression (LR), and support vector machines (SVM). In the experiments, 10-fold cross validation is chosen as the test method, and four metrics (accuracy, specificity, sensitivity, AUC) are considered to evaluate the performance. Consequently, the best accuracy of 97.29% is observed by k-NN and RF techniques. In a detailed examination, it is revealed that the most qualified technique rises as RF for the proposed model by considering the accuracy and AUC rates.

Phishing is an online threat where an attacker impersonates an authentic and trustworthy organization to obtain sensitive information from a victim. One example of such is trolling, which has long been considered a problem. However, recent advances in phishing detection, such as machine learning-based methods, have assisted in combatting these attacks. Therefore, this paper develops and compares four models for investigating the efficiency of using machine learning to detect phishing domains. It also compares the most accurate model of the four with existing solutions in the literature. These models were developed using artificial neural networks (ANNs), support vector machines (SVMs), decision trees (DTs), and random forest (RF) techniques. Moreover, the uniform resource locator’s (URL’s) UCI phishing domains dataset is used as a benchmark to evaluate the models. Our findings show that the model based on the random forest technique is the most accurate of the other four techniques and outperforms other solutions in the literature.

Phishing is an online threat where an attacker impersonates an authentic and trustworthy organization to obtain sensitive information from a victim. One example of such is trolling, which has long been considered a problem. However, recent advances in phishing detection, such as machine learning-based methods, have assisted in combatting these attacks. Therefore, this paper develops and compares four models for investigating the efficiency of using machine learning to detect phishing domains. It also compares the most accurate model of the four with existing solutions in the literature. These models were developed using artificial neural networks(ANNs), support vector machines (SVMs), decision trees (DTs), and random forest (RF)techniques. Moreover, the uniform resource locator’s (URL’s) UCI phishing domains dataset is used as a benchmark to evaluate the models. Our findings show that the model based on the random forest technique is the most accurate of the other four techniques and outperforms other solutions in the literature.

Oesophagectomy is an operation with a high risk of postoperative complications. The aim of this single-centre retrospective study was to apply machine-learning methods to predict complications (Clavien-Dindo grade IIIa or higher) and specific adverse events. Patients with resectable adenocarcinoma or squamous cell carcinoma of the oesophagus and gastro-oesophageal junction who underwent Ivor Lewis oesophagectomy between 2016 and 2021 were included. The tested algorithms were logistic regression after recursive feature elimination, random forest, k-nearest neighbour, support vector machine, and neural network. The algorithms were also compared with a current risk score (the Cologne risk score). 457 patients had Clavien-Dindo grade IIIa or higher complications (52.9 per cent) versus 407 patients with Clavien-Dindo grade 0, I, or II complications (47.1 per cent). After 3-fold imputation and 3-fold cross-validation, the overall accuracies were: logistic regression after recursive feature elimination, 0.528; random forest, 0.535; k-nearest neighbour, 0.491; support vector machine, 0.511; neural network, 0.688; and Cologne risk score, 0.510. For medical complications, the results were: logistic regression after recursive feature elimination, 0.688; random forest, 0.664; k-nearest neighbour, 0.673; support vector machine, 0.681; neural network, 0.692; and Cologne risk score, 0.650. For surgical complications, the results were: logistic regression after recursive feature elimination, 0.621; random forest, 0.617; k-nearest neighbour, 0.620; support vector machine, 0.634; neural network, 0.667; and Cologne risk score, 0.624. The calculated area under the curve of the neural network was 0.672 for Clavien-Dindo grade IIIa or higher, 0.695 for medical complications, and 0.653 for surgical complications. The neural network scored the highest accuracies compared with all of the other models for the prediction of postoperative complications after oesophagectomy.

Sepsis is a leading cause of mortality in the intensive care unit. Early prediction of sepsis can reduce the overall mortality rate and cost of sepsis treatment. Some studies have predicted mortality and development of sepsis using machine learning models. However, there is a gap between the creation of different machine learning algorithms and their implementation in clinical practice.This study utilized data from the Medical Information Mart for Intensive Care III. We established and compared the gradient boosting decision tree (GBDT), logistic regression (LR), k-nearest neighbor (KNN), random forest (RF), and support vector machine (SVM).A total of 3937 sepsis patients were included, with 34.3% mortality in the Medical Information Mart for Intensive Care III group. In our comparison of 5 machine learning models (GBDT, LR, KNN, RF, and SVM), the GBDT model showed the best performance with the highest area under the receiver operating characteristic curve (0.992), recall (94.8%), accuracy (95.4%), and F1 score (0.933). The RF, SVM, and KNN models showed better performance (area under the receiver operating characteristic curve: 0.980, 0.898, and 0.877, respectively) than the LR (0.876).The GBDT model showed better performance than other machine learning models (LR, KNN, RF, and SVM) in predicting the mortality of patients with sepsis in the intensive care unit. This could be used to develop a clinical decision support system in the future.

Gesture recognition is a topic in computer science and language technology that aims to interpret human gestures with computer programs and many different algorithms. It can be seen as the way computers can understand human body language. Today, the main interaction tools between computers and humans are still the keyboard and mouse. Gesture recognition can be used as a tool for communication with the machine and interaction without any mechanical device such as a keyboard or mouse. In this paper, we present the results of a comparison of eight different machine learning (ML) classifiers in the task of human hand gesture recognition and classification to explore how to efficiently implement one or more tested ML algorithms on an 8-bit AVR microcontroller for on-line human gesture recognition with the intention to gesturally control the mobile robot. The 8-bit AVR microcontrollers are still widely used in the industry, but due to their lack of computational power and limited memory, it is a challenging task to efficiently implement ML algorithms on them for on-line classification. Gestures were recorded by using inertial sensors, gyroscopes, and accelerometers placed at the wrist and index finger. One thousand and eight hundred (1800) hand gestures were recorded and labelled. Six important features were defined for the identification of nine different hand gestures using eight different machine learning classifiers: Decision Tree (DT), Random Forests (RF), Logistic Regression (LR), Linear Discriminant Analysis (LDA), Support Vector Machine (SVM) with linear kernel, Naïve Bayes classifier (NB), K-Nearest Neighbours (KNN), and Stochastic Gradient Descent (SGD). All tested algorithms were ranged according to Precision, Recall, and F1-score (abb.: P-R-F1). The best algorithms were SVM (P-R-F1: 0.9865, 0.9861, and 0.0863), and RF (P-R-F1: 0.9863, 0.9861, and 0.0862), but their main disadvantage is their unusability for on-line implementations in 8-bit AVR microcontrollers, as proven in the paper. The next best algorithms have had only slightly poorer performance than SVM and RF: KNN (P-R-F1: 0.9835, 0.9833, and 0.9834) and LR (P-R-F1: 0.9810, 0.9810, and 0.9810). Regarding the implementation on 8-bit microcontrollers, KNN has proven to be inadequate, like SVM and RF. However, the analysis for LR has proved that this classifier could be efficiently implemented on targeted microcontrollers. Having in mind its high F1-score (comparable to SVM, RF, and KNN), this leads to the conclusion that the LR is the most suitable classifier among tested for on-line applications in resource-constrained environments, such as embedded devices based on 8-bit AVR microcontrollers, due to its lower computational complexity in comparison with other tested algorithms.

Aim: Coronary illness is one of the main sources of mortality in this current world. A basic test in the field of clinical information assessment is the expectation of cardiovascular infection. The fundamental target of this paper is to work on the exactness of the Data mining algorithms being used to improve heart disease prediction. We will be comparing the novel Random forest with K Nearest Neighbor to find out which of these can give us the best accuracy. Material and Methods: The study used 98 samples with Novel Random Forest and K Nearest Neighbor is executed with varying training and testing splits for predicting the accuracy for heart disease prediction with the 80% of G-power value and data about cardiovascular disease was gathered from numerous web sources, including latest research findings and threshold 0.05%, confidence interval 95% mean and standard deviation. The performance of the accuracy rate of the classifiers is examined using the heart disease dataset.. There was a statistical insignificant difference among Innovative Random Forest and K Nearest Neighbor p = 0.484 (p>0.05). Results and Discussion: The accuracy of predicting heart disease in Novel Random Forest 90.16% and K Nearest Neighbor 67.21% is obtained. Conclusion: Prediction of Heart disease using the innovative Random Forest (RF) technique appears to be a significant improvement over the K Nearest Neighbor (KNN) algorithm in terms of accuracy.

In recent times, there have been increased rates at which researchers are searching for advanced ways of carrying out land-use land-cover (LULC) mapping, especially in developing countries. Four machine-learning algorithms, namely Random Forest (RF), Support Vector Machine (SVM), K-Nearest Neighbour (K-NN), and Gaussian Mixture Models (GMM) were examined. This study also attempted to validate the various models using the index-based validation method. Accuracy assessment was performed by using the Kappa coefficient. The results of the LULC showed that RF classified 23% of the study area as bare land, SVM has 24% of the study area classified as bare land, K-NN also allotted 24% to bare land, while that of GMM classifier was 30%. The overall accuracy of RF, SVM, K-NN and GMM were 0.9840, 0.9780, 0.9641 and 0.9421 respectively. The Kappa Coefficient of the various classifiers were RF (0.9695), SVM (0.9580), K-NN (0.9319) and GMM (0.8916). This study showed that though all the algorithms performed relatively very well, RF performed better than the other classifiers. It suffices to state that, there is a need for further studies since other extraneous environmental variables may be underpinning these conclusions.