A Multi-label book genre classification: Comparison of machine learning techniques and problem transformation methods

Two stage architecture for multi-label learning

The increasing prevalence of advanced technology in sports has empowered athletes, coaches, and team management to elevate their performance and fine-tune team tactics. Notably, player collaborations have emerged as a critical factor in reshaping the dynamics of matches, yet a lack of tools for accurate partnership analysis often leads to instinct-driven decisions rather than data-based insights. The aim of this study is to investigate the potential of harnessing machine learning algorithms to evaluate the significance of player partnerships in football and their potential impact on match out-comes. This study innovatively forecasts player partnership formations in live matches, demonstrating adept management of extensive, unstructured datasets to create a robust solution. It offers valuable insights for coaches to predict and build effective player partnerships, resulting in sustainable advantages for the team. The research utilizes machine learning techniques, including Logistic Regression (LR), Decision Tree (DT), K-nearest Neighbor (KNN), Support Vector Machine (SVM), Multinomial Naïve Bayes (MNB), and Random Forest (RF) classifiers. These models were applied and evaluated to construct models and assess player partnerships. Necessary data was sourced from the Kaggle dataset, which encompasses information on 167,798 player assists from 9,074 European league football matches spanning six seasons across five leagues. Machine learning models underwent training and evaluation through an 80 − 20 data split and further verified through cross-validation to ensure robustness across league datasets. Balanced sampling and k-fold cross-validation were used to mitigate data imbalance and validate model generalization. Notably, the SVM, DT and RF models demonstrated impressive accuracy of 99%, with the highest F1-score of 0.99, encompassing all football leagues. The SVM and RF models also achieved the highest precision, at 0.99, albeit with a relatively longer computation time of up to 60 s. Model performance varied marginally across leagues (± 0.3% standard deviation in accuracy and F1-score), indicating stable predictive behaviour across gameplay variations. Unlike prior studies that analyzed team synchronization or motion dynamics, this work uniquely models player partnerships based on assist-goal frequency patterns and forecasts their continuity across matches using machine learning classifiers. This study provides valuable insights for devising strategies that can confer a competitive edge in matches. The findings offer a practical means for team managers and coaches to make well-informed decisions regarding player partnerships.

Developing a multi-label learning model to predict major adverse cardiovascular events in patients with unstable angina pectoris: A prospective cohort study.

Machine learning involves the task of training systems to be able to make decisions without being explicitly programmed. Important among machine learning tasks is classification involving the process of training machines to make predictions from predefined labels. Classification is broadly categorized into three distinct groups: single-label (SL), multi-class, and multi-label (ML) classification. This research work presents an application of a multi-label classification (MLC) technique in automating Quranic verses labeling. MLC has been gaining attention in recent years. This is due to the increasing amount of works based on real-world classification problems of multi-label data. In traditional classification problems, patterns are associated with a single-label from a set of disjoint labels. However, in MLC, an instance of data is associated with a set of labels. In this paper, three standard <em>MLC</em> methods: <span>binary relevance (BR), classifier chain (CC), and label powerset (LP) algorithms are implemented with four baseline classifiers: support vector machine (SVM), naïve Bayes (NB), k-nearest neighbors (k-NN), and J48. The research methodology adopts the multi-label problem transformation (PT) approach. The results are validated using six conventional performance metrics. These include: hamming loss, accuracy, one error, micro-F1, macro-F1, and avg. precision. From the results, the classifiers effectively achieved above 70% accuracy mark. Overall, SVM achieved the best results with CC and LP algorithms.</span>

Electronic commerce (e-commerce) brings huge advantages to businesses for selling products through multiple online shops. However, companies have difficulties in supervising the prices of products set by different retail shops on e-commerce platforms. Addressing these difficulties, we suggest a method to identify and predict products that sell at incorrect prices using a machine learning model combined price analysis. The study uses four machine learning models: K-nearest Neighbor (KNN), Random Forest (RF), Support Vector Machine (SVM), and Multinomial Naive Bayes (MNB) and two text-based information extraction methods: BoW and TF-IDF to find to the best method. The research results show that the RF model and text-based information extraction method by the BoW provide more average accuracy than other specific models, when experimenting on the filter dataset the average accuracy after 10 runs are RF: 98.06%, SVM: 83.92%, MNB: 92.21%, KNN: 94.06%. Experimental results on the product dataset have an accuracy of RF: 83.02%, SVM: 55%, MNB: 79.33%, KNN: 79.36%.

Multi-label classification task has many applications in Text Categorization, Multimedia, Biology, Chemical data analysis and Social Network Mining, among others. Different approaches have been developed: Binary Relevance (BR), Label Power Set (LPS), Random k label sets (RAkEL), some of them consider the interaction between labels in a chain (Chain Classifier) and other alternatives around this method are derived, for instance, Probabilistic Chain Classifier, Monte Carlo Chain Classifier and Bayesian Chain Classifier (BCC). All previous approaches have in common and focus on is in considering different orders or combinations of the way the labels have to be predicted. Given that feature selection has proved to be important in classification tasks, reducing the dimensionality of the problem and even improving classification model’s accuracy. In this work a feature selection technique is tested in BCC algorithm with two searching methods, one using Best First (BF-FS-BCC) and another with GreedyStepwise (GS-FS-BCC), these methods are compared, the winner is also compared with BCC, both tests are compared through Wilcoxon Signed Rank test, in addition it is compared with others Chain Classifier and finally it is compared with others approaches (BR, RAkEL, LPS).

Integration of feature vector selection and support vector machine for classification of imbalanced data

Forests hold vital ecological significance, and the ability to accurately classify tree species is integral to conservation and management practices. This research investigates the application of machine learning techniques to airborne Light Detection and Ranging (LiDAR) data for the multi-class classification of tree species, specifically Alder, Aspen, Birch, Fir, Pine, Spruce, and Tilia. High-density LiDAR data from varied forest landscapes were subjected to a rigorous preprocessing and noise reduction protocol, followed by feature extraction to discern structural characteristics indicative of species identity. We assessed the performance of six machine learning models: Logistic Regression, Decision Tree, Random Forest, Support Vector Classifier (SVC), k-Nearest Neighbors (KNN), and Gradient Boosting. The analysis was based on metrics of accuracy, precision, recall, and F1 score. Logistic Regression and Random Forest models outperformed others, achieving accuracies of 0.81, precision of 0.80, recall of 0.81, and an F1 score of 0.80. In contrast, the KNN algorithm had the lowest accuracy of 0.60, precision and recall of 0.60, and an F1 score of 0.59. These results demonstrate the robustness of Logistic Regression and Random Forest for classifying complex LiDAR datasets. The study underscores the potential of these models to support ecological monitoring, enhance forest management, and aid in biodiversity conservation. Future research directions include the fusion of LiDAR data with other environmental variables, application of deep learning for improved feature extraction, and validation of the models across broader species and geographical ranges. This research marks a significant step towards leveraging advanced machine learning to interpret and utilize LiDAR data for environmental and ecological applications.

Bond rating is regarded as an important event for measuring financial risk of companies and for determining the investment returns of investors. As a result, it has been a popular research topic for researchers to predict companies' credit ratings by applying statistical and machine learning techniques. The statistical techniques, including multiple regression, multiple discriminant analysis (MDA), logistic models (LOGIT), and probit analysis, have been traditionally used in bond rating. However, one major drawback is that it should be based on strict assumptions. Such strict assumptions include linearity, normality, independence among predictor variables and pre-existing functional forms relating the criterion variablesand the predictor variables. Those strict assumptions of traditional statistics have limited their application to the real world. Machine learning techniques also used in bond rating prediction models include decision trees (DT), neural networks (NN), and Support Vector Machine (SVM). Especially, SVM is recognized as a new and promising classification and regression analysis method. SVM learns a separating hyperplane that can maximize the margin between two categories. SVM is simple enough to be analyzed mathematical, and leads to high performance in practical applications. SVM implements the structuralrisk minimization principle and searches to minimize an upper bound of the generalization error. In addition, the solution of SVM may be a global optimum and thus, overfitting is unlikely to occur with SVM. In addition, SVM does not require too many data sample for training since it builds prediction models by only using some representative sample near the boundaries called support vectors. A number of experimental researches have indicated that SVM has been successfully applied in a variety of pattern recognition fields. However, there are three major drawbacks that can be potential causes for degrading SVM's performance. First, SVM is originally proposed for solving binary-class classification problems. Methods for combining SVMs for multi-class classification such as One-Against-One, One-Against-All have been proposed, but they do not improve the performance in multi-class classification problem as much as SVM for binary-class classification. Second, approximation algorithms (e.g. decomposition methods, sequential minimal optimization algorithm) could be used for effective multi-class computation to reduce computation time, but it could deteriorate classification performance. Third, the difficulty in multi-class prediction problems is in data imbalance problem that can occur when the number of instances in one class greatly outnumbers the number of instances in the other class. Such data sets often cause a default classifier to be built due to skewed boundary and thus the reduction in the classification accuracy of such a classifier. SVM ensemble learning is one of machine learning methods to cope with the above drawbacks. Ensemble learning is a method for improving the performance of classification and prediction algorithms. AdaBoost is one of the widely used ensemble learning techniques. It constructs a composite classifier by sequentially training classifiers while increasing weight on the misclassified observations through iterations. The observations that are incorrectly predicted by previous classifiers are chosen more often than examples that are correctly predicted. Thus Boosting attempts to produce new classifiers that are better able to predict examples for which the current ensemble's performance is poor. In this way, it can reinforce the training of the misclassified observations of the minority class. This paper proposes a multiclass Geometric Mean-based Boosting (MGM-Boost) to resolve multiclass prediction problem. Since MGM-Boost introduces the notion of geometric mean into AdaBoost, it can perform learning process considering the geometric mean-based accuracy and errors of multiclass. This study applies MGM-Boost to the real-world bond rating case for Korean companies to examine the feasibility of MGM-Boost. 10-fold cross validations for threetimes with different random seeds are performed in order to ensure that the comparison among three different classifiers does not happen by chance. For each of 10-fold cross validation, the entire data set is first partitioned into tenequal-sized sets, and then each set is in turn used as the test set while the classifier trains on the other nine sets. That is, cross-validated folds have been tested independently of each algorithm. Through these steps, we have obtained the results for classifiers on each of the 30 experiments. In the comparison of arithmetic mean-based prediction accuracy between individual classifiers, MGM-Boost (52.95%) shows higher prediction accuracy than both AdaBoost (51.69%) and SVM (49.47%). MGM-Boost (28.12%) also shows the higher prediction accuracy than AdaBoost (24.65%) and SVM (15.42%)in terms of geometric mean-based prediction accuracy. T-test is used to examine whether the performance of each classifiers for 30 folds is significantly different. The results indicate that performance of MGM-Boost is significantly different from AdaBoost and SVM classifiers at 1% level. These results mean that MGM-Boost can provide robust and stable solutions to multi-classproblems such as bond rating.

There has been recent interest in the application of machine learning techniques to neuroimaging-based diagnosis. These methods promise fully automated, standard PC-based clinical decisions, unbiased by variable radiological expertise. We recently used support vector machines (SVMs) to separate sporadic Alzheimer's disease from normal ageing and from fronto-temporal lobar degeneration (FTLD). In this study, we compare the results to those obtained by radiologists. A binary diagnostic classification was made by six radiologists with different levels of experience on the same scans and information that had been previously analysed with SVM. SVMs correctly classified 95% (sensitivity/specificity: 95/95) of sporadic Alzheimer's disease and controls into their respective groups. Radiologists correctly classified 65–95% (median 89%; sensitivity/specificity: 88/90) of scans. SVM correctly classified another set of sporadic Alzheimer's disease in 93% (sensitivity/specificity: 100/86) of cases, whereas radiologists ranged between 80% and 90% (median 83%; sensitivity/specificity: 80/85). SVMs were better at separating patients with sporadic Alzheimer's disease from those with FTLD (SVM 89%; sensitivity/specificity: 83/95; compared to radiological range from 63% to 83%; median 71%; sensitivity/specificity: 64/76). Radiologists were always accurate when they reported a high degree of diagnostic confidence. The results show that well-trained neuroradiologists classify typical Alzheimer's disease-associated scans comparable to SVMs. However, SVMs require no expert knowledge and trained SVMs can readily be exchanged between centres for use in diagnostic classification. These results are encouraging and indicate a role for computerized diagnostic methods in clinical practice.

Comparison of base classifiers for multi-label learning

bstract: Malaria disease is the number one cause of death all over the Sub-Sahara world. Data mining can help extract valuable knowledge from available data in the healthcare sector. This allows training a patient health prediction model faster than in a clinical trial. Various implementation of machine learning algorithms such as Bayesian Theorem, Logistic Regression, K-Nearest Neighbor, Support Vector Machine and Multinomial Naïve Bayes (MNB), etc. have been applied on Public Hospital Malaria Disease datasets but there has been a limit to modeling using Multinomial Naïve Bayes Algorithm. This research applied MNB modeling to discover the relationship between 15 relevant attributes of the Public Hospitals data collected from Bwari General Hospital in Bwari Area Council and Maitama Hospital in Abuja Municipal Area Council, Abuja, FCT, and Nigeria. The goal is to examine how dependencies between attributes affect the performance of the classifier. The MNB produces a reliable and transparent graphical representation between the attributes with the ability to predict new scenarios. The model has an accuracy of 97%. It was concluded that the model outperformed the GNB classifier which has an accuracy of 100% and RF which also has an accuracy of 100%.

Improving autocoding performance of rare categories in injury classification: Is more training data or filtering the solution?

The availability of the data has increased tremendously due to the excess usage of social media platforms like Twitter and Facebook. Due to the abundant availability of data, scientists, businesses, educationalists and other people working under different roles have started using Sentiment Analysis (SA) to get in-depth knowledge about the sentiments of the people regarding any topic of interest. There are many techniques to implement SA, and one of them is Machine Learning (ML). This study is focused on the comparison of ancient ML methods such as Naïve Bayes (NB), Decision Tree (DT), Support Vector Machine (SVM), and a modern method, i.e., Deep Learning (DL). The ML techniques are applied to a single dataset to compare their performance in terms of accuracy to understand how they perform against each other. The study found that DL performed the best with 96.41% accuracy followed by NB and SVM with 87.18% and 82.05% respectively. DT performed the poorest with 68.21% accuracy.

Two machine learning techniques were developed to extract human settlements from very high resolution (VHR) satellite images of 3 provinces in Afghanistan: Logar, Panjsher, and Wardak. The results were then compared with analyst verified reference data information known as the LandScan Settlement Layer (LandScan SL).[1] This study attempts to compare settlement mapping results from a support vector machine (SVM) classifier specifically integrated in a current settlement mapping framework and a deep learner utilizing a convolutional neural network (CNN) approach. By comparing the results from the SVM and the CNN to the reference data information we demonstrate that the CNN yields more accurate results overall, in terms of overall pixel cells, and the SVM performs more accurately in omission, based on derived statistics against the reference data information.