Analytics of machine learning-based algorithms for text classification

Text mining is a special case of data mining which explore unstructured or semi-structured text documents, to establish valuable patterns and rules that indicate trends and significant features about specific topics. Text mining has been in pattern recognition, predictive studies, sentiment analysis and statistical theories in many areas of research, medicine, financial analysis, social life analysis, and business intelligence. Text mining uses concept of natural language processing and machine learning. Machine learning algorithms have been used and reported to give great results, but their performance of machine learning algorithms is affected by factors such as dataset domain, number of classes, length of the corpus, and feature selection techniques used. Redundant attribute affects the performance of the classification algorithm, but this can be reduced by using different feature selection techniques and dimensionality reduction techniques. Feature selection is a data preprocessing step that chooses a subset of input variable while eliminating features with little or no predictive information. Feature selection techniques are Information gain, Term Frequency, Term Frequency-Inverse document frequency, Mutual Information, and Chi-Square, which can use a filters, wrappers, or embedded approaches. To get the most value from machine learning, pairing the best algorithms with the right tools and processes is necessary. Little research has been done on the effect of feature selection techniques on classification accuracy for pairing of these algorithms with the best feature selection techniques for optimal results. In this research, a text classification experiment was conducted using incident management dataset, where incidents were classified into their resolver groups. Support vector machine (SVM), K-Nearest Neighbors (KNN), Naïve Bayes (NB) and Decision tree (DT) machine learning algorithms were examined. Filtering approach was used on the feature selection techniques, with different ranking indices applied for optimal feature set and classification accuracy results analyzed. The classification accuracy results obtained using TF were, 88% for SVM, 70% for NB, 79% for Decision tree, and KNN had 55%, while Boolean registered 90%, 83%, 82% and 75%, for SVM, NB, DT, and KNN respectively. TF-IDF, had 91%, 83%, 76%, and 56% for SVM, NB, DT, and KNN respectively. The results showed that algorithm performance is affected by feature selection technique applied. SVM performed best, followed by DT, KNN and finally NB. In conclusion, presence of noisy data leads to poor learning performance and increases the computational time. The classifiers performed differently depending on the feature selection technique applied. For optimal results, the classifier that performed best together with the feature selection technique with the best feature subset should be applied for all types of data for accurate classification performance. Keywords: Text Classification, Supervised Machine Learning, Feature Selection DOI : 10.7176/JIEA/9-3-06 Publication date :May 31 st 2019

Machine learning methods, including Deep learnings are popular for data processing. Deep learning methods have shown great promise in applied in natural language processing (NLP) tasks. Text classification is a supervised machine learning task that involves labelled documents to train classifier. Previous works involving machine learning and deep learning methods for text classification have been tested with relatively small- sized data instances. In this paper, we compared the performance of the machine leaning and deep learning algorithms in text classification task. This paper also studied, explored and compared the scalability of these methods with respect to bigger data instances. We used support vector machines (SVM), Logistic regression, Random forest and Naive Bayes Machine leaning algorithms and convolutional neural network (CNN) deep learning method. The task involved a multi-class classification problem involving six (6) classes consisting of six thousand (6,000) data instances with an average of 20 sentences in each data instance. The CNN deep leaning algorithm outperformed all the machine learning algorithms, achieving an accuracy of over 85%. This is because the filter weights are leaning are updated in backward propagation in each epoch, hence, this result in better result compared to the traditional methods.

Sentiment analysis plays a pivotal role in the hotel industry, where user-generated reviews significantly influence customer decisions. However, traditional machine learning (ML) methods often struggle with the linguistic nuances of languages such as Thai. This study investigates the effectiveness of fine-tuning WangchanBERTa, a monolingual Thai large language model (LLM), for sentiment classification of hotel reviews from Bangkok. The model's performance was compared with ML algorithms, including extreme gradient boosting (XGBoost), support vector machines (SVM), logistic regression (LR), and multinomial naïve Bayes (MNB). The comparison highlights the advantages of deep contextual understanding enabled by transformer-based architecture. To improve interpretability, Shapley Additive Explanation (SHAP) was applied to the best-performing model to analyze feature importance. The results show that the fine-tuned LLM outperformed all ML models, achieving over 92% across all evaluation metrics (accuracy, precision, recall, and F1-score). SHAP analysis enhanced transparency by revealing sentiment drivers relevant to the hotel domain. This study contributes to the advancement of natural language processing (NLP) for low-resource languages by demonstrating the effectiveness of domain-specific fine-tuning combined with explainable artificial intelligence (XAI) in practical applications.

In today's digital world, app stores have become an essential part of software distribution, providing customers with a wide range of applications and opportunities for software developers to showcase their work. This study elaborates on the importance of end-user feedback for software evolution. However, in the literature, more emphasis has been given to high-rating & popular software apps while ignoring comparatively low-rating apps. Therefore, the proposed approach focuses on end-user reviews collected from 64 low-rated apps representing 14 categories in the Amazon App Store. We critically analyze feedback from low-rating apps and developed a grounded theory to identify various concepts important for software evolution and improving its quality including user interface (UI) and user experience (UX), functionality and features, compatibility and device-specific, performance and stability, customer support and responsiveness and security and privacy issues. Then, using a grounded theory and content analysis approach, a novel research dataset is curated to evaluate the performance of baseline machine learning (ML), and state-of-the-art deep learning (DL) algorithms in automatically classifying end-user feedback into frequently occurring issues. Various natural language processing and feature engineering techniques are utilized for improving and optimizing the performance of ML and DL classifiers. Also, an experimental study comparing various ML and DL algorithms, including multinomial naive Bayes (MNB), logistic regression (LR), random forest (RF), multi-layer perception (MLP), k-nearest neighbors (KNN), AdaBoost, Voting, convolutional neural network (CNN), long short-term memory (LSTM), bidirectional long short term memory (BiLSTM), gated recurrent unit (GRU), bidirectional gated recurrent unit (BiGRU), and recurrent neural network (RNN) classifiers, achieved satisfactory results in classifying end-user feedback to commonly occurring issues. Whereas, MLP, RF, BiGRU, GRU, CNN, LSTM, and Classifiers achieved average accuracies of 94%, 94%, 92%, 91%, 90%, 89%, and 89%, respectively. We employed the SHAP approach to identify the critical features associated with each issue type to enhance the explainability of the classifiers. This research sheds light on areas needing improvement in low-rated apps and opens up new avenues for developers to improve software quality based on user feedback.

bstract: Malaria disease is the number one cause of death all over the Sub-Sahara world. Data mining can help extract valuable knowledge from available data in the healthcare sector. This allows training a patient health prediction model faster than in a clinical trial. Various implementation of machine learning algorithms such as Bayesian Theorem, Logistic Regression, K-Nearest Neighbor, Support Vector Machine and Multinomial Naïve Bayes (MNB), etc. have been applied on Public Hospital Malaria Disease datasets but there has been a limit to modeling using Multinomial Naïve Bayes Algorithm. This research applied MNB modeling to discover the relationship between 15 relevant attributes of the Public Hospitals data collected from Bwari General Hospital in Bwari Area Council and Maitama Hospital in Abuja Municipal Area Council, Abuja, FCT, and Nigeria. The goal is to examine how dependencies between attributes affect the performance of the classifier. The MNB produces a reliable and transparent graphical representation between the attributes with the ability to predict new scenarios. The model has an accuracy of 97%. It was concluded that the model outperformed the GNB classifier which has an accuracy of 100% and RF which also has an accuracy of 100%.

Recent advances in machine learning techniques have significantly influenced the development of early diabetes detection systems. In this paper, different machine learning (ML) algorithms are employed for diabetes detection based on hand movement. The spatial and temporal information for horizontal and vertical hand movement deviations for both diabetic and non-diabetic subjects are obtained using a data acquisition system. A dataset is constructed based on the obtained hand movement information along with other related attributes such as age, gender, weight and vision health. Binary classification of diabetic and non-diabetic subjects is achieved using several machine learning algorithms. These algorithms include Naïve Bayes (NB), Logistic Regression (LR), Linear Discriminate Analysis (LDA), Classification and Regression Tree (CART), k-Nearest Neighbors (KNN), Support Vector Machine (SVM) and Multi-layer Perceptron (MLP). The considered ML algorithms are evaluated and ranked according to a set of key performance metrics that include accuracy, precision, recall and F-1 score. Algorithm ranking results demonstrate that LR, CART, KNN and MLP algorithms have achieved higher ranking positions as compared to other algorithms. The best ranked ML algorithms are then validated and thoroughly investigated to prove their ability to discriminate between diabetic and non-diabetic subjects. Validation results show that the best-ranked ML algorithms have achieved high classification accuracy, precision, recall and F-1 scores. Therefore, the considered machine learning algorithms can serve as efficient tools for diabetes detection based on the constructed dataset.

The increasing prevalence of advanced technology in sports has empowered athletes, coaches, and team management to elevate their performance and fine-tune team tactics. Notably, player collaborations have emerged as a critical factor in reshaping the dynamics of matches, yet a lack of tools for accurate partnership analysis often leads to instinct-driven decisions rather than data-based insights. The aim of this study is to investigate the potential of harnessing machine learning algorithms to evaluate the significance of player partnerships in football and their potential impact on match out-comes. This study innovatively forecasts player partnership formations in live matches, demonstrating adept management of extensive, unstructured datasets to create a robust solution. It offers valuable insights for coaches to predict and build effective player partnerships, resulting in sustainable advantages for the team. The research utilizes machine learning techniques, including Logistic Regression (LR), Decision Tree (DT), K-nearest Neighbor (KNN), Support Vector Machine (SVM), Multinomial Naïve Bayes (MNB), and Random Forest (RF) classifiers. These models were applied and evaluated to construct models and assess player partnerships. Necessary data was sourced from the Kaggle dataset, which encompasses information on 167,798 player assists from 9,074 European league football matches spanning six seasons across five leagues. Machine learning models underwent training and evaluation through an 80 − 20 data split and further verified through cross-validation to ensure robustness across league datasets. Balanced sampling and k-fold cross-validation were used to mitigate data imbalance and validate model generalization. Notably, the SVM, DT and RF models demonstrated impressive accuracy of 99%, with the highest F1-score of 0.99, encompassing all football leagues. The SVM and RF models also achieved the highest precision, at 0.99, albeit with a relatively longer computation time of up to 60 s. Model performance varied marginally across leagues (± 0.3% standard deviation in accuracy and F1-score), indicating stable predictive behaviour across gameplay variations. Unlike prior studies that analyzed team synchronization or motion dynamics, this work uniquely models player partnerships based on assist-goal frequency patterns and forecasts their continuity across matches using machine learning classifiers. This study provides valuable insights for devising strategies that can confer a competitive edge in matches. The findings offer a practical means for team managers and coaches to make well-informed decisions regarding player partnerships.

Gesture recognition is a topic in computer science and language technology that aims to interpret human gestures with computer programs and many different algorithms. It can be seen as the way computers can understand human body language. Today, the main interaction tools between computers and humans are still the keyboard and mouse. Gesture recognition can be used as a tool for communication with the machine and interaction without any mechanical device such as a keyboard or mouse. In this paper, we present the results of a comparison of eight different machine learning (ML) classifiers in the task of human hand gesture recognition and classification to explore how to efficiently implement one or more tested ML algorithms on an 8-bit AVR microcontroller for on-line human gesture recognition with the intention to gesturally control the mobile robot. The 8-bit AVR microcontrollers are still widely used in the industry, but due to their lack of computational power and limited memory, it is a challenging task to efficiently implement ML algorithms on them for on-line classification. Gestures were recorded by using inertial sensors, gyroscopes, and accelerometers placed at the wrist and index finger. One thousand and eight hundred (1800) hand gestures were recorded and labelled. Six important features were defined for the identification of nine different hand gestures using eight different machine learning classifiers: Decision Tree (DT), Random Forests (RF), Logistic Regression (LR), Linear Discriminant Analysis (LDA), Support Vector Machine (SVM) with linear kernel, Naïve Bayes classifier (NB), K-Nearest Neighbours (KNN), and Stochastic Gradient Descent (SGD). All tested algorithms were ranged according to Precision, Recall, and F1-score (abb.: P-R-F1). The best algorithms were SVM (P-R-F1: 0.9865, 0.9861, and 0.0863), and RF (P-R-F1: 0.9863, 0.9861, and 0.0862), but their main disadvantage is their unusability for on-line implementations in 8-bit AVR microcontrollers, as proven in the paper. The next best algorithms have had only slightly poorer performance than SVM and RF: KNN (P-R-F1: 0.9835, 0.9833, and 0.9834) and LR (P-R-F1: 0.9810, 0.9810, and 0.9810). Regarding the implementation on 8-bit microcontrollers, KNN has proven to be inadequate, like SVM and RF. However, the analysis for LR has proved that this classifier could be efficiently implemented on targeted microcontrollers. Having in mind its high F1-score (comparable to SVM, RF, and KNN), this leads to the conclusion that the LR is the most suitable classifier among tested for on-line applications in resource-constrained environments, such as embedded devices based on 8-bit AVR microcontrollers, due to its lower computational complexity in comparison with other tested algorithms.

Sentiment analysis is one of the major topics of natural language processing which is used to determine whether data is positive, negative or neutral. It is often performed on textual data to help businesses monitor brand and product sentiment in customer feedback to understand their customers’ needs. This paper explores various machine learning algorithms including Logistic Regression (LR), Multinomial Naïve Bayes (MNB), Support Vector Classifier (SVC), Multi-layer Perceptron Classifier (MLP) and Extreme Gradient Boosting Classifier (XGB) to build sentiment analysis models tailored for the gaming domain to classify reviews into positive, negative and neutral. The models were trained on game reviews obtained from Metacritic and Steam. Various data preprocessing and model optimization techniques have been employed and the performance of the models were evaluated and compared. SVC has been determined as the best-performing model among all the models. Keywords: Sentiment Analysis, Natural Language Processing, Machine Learning, Support Vector Machine, Game Reviews

Improving autocoding performance of rare categories in injury classification: Is more training data or filtering the solution?

We can state undoubtedly that Bangla language is rich enough to work with and implement various Natural Language Processing (NLP) tasks. Though it needs proper attention, hardly NLP field has been explored with it. In this age of digitalization, large amount of Bangla news contents are generated in online platforms. Some of the contents are inappropriate for the children or aged people. With the motivation to filter out news contents easily, the aim of this work is to perform document level sentiment analysis (SA) on Bangla online news. In this respect, the dataset is created by collecting news from online Bangla newspaper archive. Further, the documents are manually annotated into positive and negative classes. Composite process technique of “Pipeline” class including Count Vectorizer, transformer (TF-IDF) and machine learning (ML) classifiers are employed to extract features and to train the dataset. Six supervised ML classifiers (i.e. Multinomial Naive Bayes (MNB), K-Nearest Neighbor (K-NN), Random Forest (RF), (C4.5) Decision Tree (DT), Logistic Regression (LR) and Linear Support Vector Machine (LSVM)) are used to analyze the best classifier for the proposed model. There has been very few works on SA of Bangla news. So, this work is a small attempt to contribute in this field. This model showed remarkable efficiency through better results in both the validation process of percentage split method and 10-fold cross validation. Among all six classifiers, RF has outperformed others by 99% accuracy. Even though LSVM has shown lowest accuracy of 80%, it is also considered as good output. However, this work has also exhibited surpassing outcome for recent and critical Bangla news indicating proper feature extraction to build up the model.

The coronavirus disease (COVID-19) was first identified in Wuhan, China, in December 2019. Based on the Indonesian COVID-19 Gugus Tugas report on July 14, 2021. This virus has infected 2.6 million people and has claimed the lives of 69,210 Indonesians. The spread of COVID-19 and the dangers it causes have prompted the government to take a policy to initiate the vaccination movement immediately. Currently, in Indonesia, the vaccination program is still running, but the response from the community is still pros and cons. This study aims to determine the results of Indonesian public opinion after the vaccination program runs by conducting sentiment analysis. The dataset used is Youtube commentary data, taken from the YouTube channel of the Indonesian Ministry of Health, the Presidential Secretariat, President Joko Widodo, and Najwa Shihab, using the keyword ‘Covid-19 Vaccine’. Comment data is retrieved from November 2020 to July 2021. The crawling of the data is done manually using the access token received from the YouTube API and the python programming language to extract the requested information data. Comment data is divided into three categories: positive, negative, and neutral. To find the best classification model, applying several machine learning algorithms, namely: K-nearest neighbor (KNN), Support Vector Machine (SVM), Multinomial Naive Bayes (MNB), Logistic Regression (LR), Random Forest (RF), Decision Tree (DT), Adaboost, and Stochastic gradient descent (SGD). The results of sentiment analysis research related to COVID-19 tend to get negative responses with sentiment measurements obtained more than 4.6 thousand positive comments (44.9%), more than 3.4 thousand positive comments (33.3%), and the rest more than 2.2 thousand. neutral comments (21.8%). The best accuracy value was obtained with the Linear Regression (LR) algorithm of 68%.

Periodontitis is increasingly associated with heart failure, and the goal of this study was to develop and validate a prediction model based on machine learning algorithms for the risk of heart failure in middle-aged and elderly participants with periodontitis. We analyzed data from a total of 2876 participants with a history of periodontitis from the National Health and Nutrition Examination Survey (NHANES) 2009 to 2014, with a training set of 1980 subjects with periodontitis from the NHANES 2009 to 2012 and an external validation set of 896 subjects from the NHANES 2013 to 2014. The independent risk factors for heart failure were identified using univariate and multivariate logistic regression analysis. Machine learning algorithms such as logistic regression, k-nearest neighbor, support vector machine, random forest, gradient boosting machine, and multilayer perceptron were used on the training set to construct the models. The performance of the machine learning models was evaluated using 10-fold cross-validation on the training set and receiver operating characteristic curve (ROC) analysis in the validation set. Based on the results of univariate logistic regression and multivariate logistic regression, it was found that age, race, myocardial infarction, and diabetes mellitus status were independent predictors of the risk of heart failure in participants with periodontitis. Six machine learning models, including logistic regression, K-nearest neighbor, support vector machine, random forest, gradient boosting machine, and multilayer perceptron, were built on the training set, respectively. The area under the ROC for the 6 models was obtained using 10-fold cross-validation with values of 0 848, 0.936, 0.859, 0.889, 0.927, and 0.666, respectively. The areas under the ROC on the external validation set were 0.854, 0.949, 0.647, 0.933, 0.855, and 0.74, respectively. K-nearest neighbor model got the best prediction performance across all models. Out of 6 machine learning models, the K-nearest neighbor algorithm model performed the best. The prediction model offers early, individualized diagnosis and treatment plans and assists in identifying the risk of heart failure occurrence in middle-aged and elderly patients with periodontitis.

ObjectiveEnhanced technology in computer and internet has driven scale and quality of data to be improved in various areas including healthcare sectors. Machine Learning (ML) has played a pivotal role in efficiently analyzing those big data, but a general misunderstanding of ML algorithms still exists in applying them (e.g., ML techniques can settle a problem of small sample size, or deep learning is the ML algorithm). This paper reviewed the research of diagnosing mental illness using ML algorithm and suggests how ML techniques can be employed and worked in practice.MethodsResearches about mental illness diagnostic using ML techniques were carefully reviewed. Five traditional ML algorithms-Support Vector Machines (SVM), Gradient Boosting Machine (GBM), Random Forest, Naïve Bayes, and K-Nearest Neighborhood (KNN)-frequently used for mental health area researches were systematically organized and summarized.ResultsBased on literature review, it turned out that Support Vector Machines (SVM), Gradient Boosting Machine (GBM), Random Forest, Naïve Bayes, and K-Nearest Neighborhood (KNN) were frequently employed in mental health area, but many researchers did not clarify the reason for using their ML algorithm though every ML algorithm has its own advantages. In addition, there were several studies to apply ML algorithms without fully understanding the data characteristics.ConclusionResearchers using ML algorithms should be aware of the properties of their ML algorithms and the limitation of the results they obtained under restricted data conditions. This paper provides useful information of the properties and limitation of each ML algorithm in the practice of mental health.

Football (also known as Soccer), boasts a staggering fan base of 3.5 billion individuals spread across 200 countries, making it the world’s most beloved sport. The widespread adoption of advanced technology in sports has become increasingly prominent, empowering players, coaches, and team management to enhance their performance and refine team strategies. Among these advancements, player substitution plays a crucial role in altering the dynamics of a match. However, due to the absence of proven methods or software capable of accurately predicting substitutions, these decisions are often based on instinct rather than concrete data. The purpose of this research is to explore the potential of employing machine learning algorithms to predict substitutions in Football, and how it could influence the outcome of a match. This study investigates the effect of timely and tactical substitutions in football matches and their influence on the match results. Machine learning techniques such as Logistic Regression (LR), Decision tree (DT), K-nearest Neighbor (KNN), Support Vector Machine (SVM), Multinomial Naïve Bayes (MNB), Random Forest (RF) classifiers were implemented and tested to develop models and to predict player substitutions. Relevant data was collected from the Kaggle dataset, which contains data of 51,738 substitutions from 9074 European league football matches in 5 leagues spanning 6 seasons. Machine learning models were trained and tested using an 80-20 data split and it was observed that RF model provided the best accuracy of over 70% and the best F1-score of 0.65 on the test set across all football leagues. SVM model achieved the best Precision of almost 0.8. However, the worst computation time of up to 2 min was consumed. LR showed some overfitting issues with 100% accuracy in the training set, but only 60% accuracy was obtained for the test set. To conclude, based on the time of substitution and match score-line, it was possible to predict the players who can be substituted, which can provide a match advantage. The achieved results provided an effective way to decide on player substitutions for both the team manager and coaches.