A Novel Heuristic Text Classification Algorithm Based on Support Vector Machines

The Kampus Merdeka program, launched by the Indonesian Ministry of Education, Culture, Research, and Technology in 2020, aims to enhance students' skills through hands-on work experience. Considering the rising significance of social media, particularly Twitter, in gauging public opinion, this research focuses on analyzing the sentiment towards the Kampus Merdeka program. The primary objective is to classify the sentiments expressed in tweets related to the program and compare two feature extraction techniques—TF-IDF and FastText—to identify the best approach for transforming text data into numerical vectors. The sentiment classification model was built using the Support Vector Machine (SVM) algorithm, a machine learning technique known for its accuracy in text classification. A total of 16,730 tweets were collected and analyzed, yielding an accuracy of 73% for FastText and 72% for TF-IDF. Results show that FastText is more effective in capturing semantic relationships, leading to higher accuracy in sentiment classification. Findings indicate that the public sentiment towards the Kampus Merdeka program is predominantly positive (60.7%), with negative and neutral sentiments at 33.5% and 5.8%, respectively. The success of the FastText method underscores the importance of advanced feature extraction techniques in text classification. The novelty of this research lies in its use of FastText for educational policy evaluation, providing a new perspective on using sentiment analysis to assess public perception of educational programs.

Big multimedia data is heterogeneous in essence, that is, the data may be a mixture of video, audio, text, and images. This is due to the prevalence of novel applications in recent years, such as social media, video sharing, and location based services (LBS), etc. In many multimedia applications, for example, video/image tagging and multimedia recommendation, text classification techniques have been used extensively to facilitate multimedia data processing. In this paper, we give a comprehensive review on feature selection techniques for text classification. We begin by introducing some popular representation schemes for documents, and similarity measures used in text classification. Then, we review the most popular text classifiers, including Nearest Neighbor (NN) method, Naive Bayes (NB), Support Vector Machine (SVM), Decision Tree (DT), and Neural Networks. Next, we survey four feature selection models, namely the filter, wrapper, embedded and hybrid, discussing pros and cons of the state-of-the-art feature selection approaches. Finally, we conclude the paper and give a brief introduction to some interesting feature selection work that does not belong to the four models.

Prediction of cause of death from forensic autopsy reports using text classification techniques: A comparative study

How to Improve Text Summarization and Classification by Mutual Cooperation on an Integrated Framework

Text classification is a classical problem of natural language processing where given texts are classified into relevant classes. Text classification techniques are extensively used in finding news categories, search engine optimization, automated textual response and many more. With the exponential growth of digital data, it is inevitable that an automatic classification of documents is needed for efficient information retrieval and document archival. Over the years, text classification problems are approached by using classical machine learning techniques like naive bayes(NB), support vector machine(SVM), logistic regression(LR), random forest(RF) and others. In this article, an Attention based deep learning (DL) model is proposed and applied to the news corpus and then the results are compared with the results of classical machine learning models. A comparative analysis of classification accuracy of the DL model with prominent models like LR, RF and NB are presented in this work.

Supervised machine learning studies are gaining more significant recently because of the availability of the increasing number of the electronic documents from different resources. Text classification can be defined that the task was automatically categorized a group documents into one or more predefined classes according to their subjects. Thereby, the major objective of text classification is to enable users for extracting information from textual resource and deals with process such as retrieval, classification, and machine learning techniques together in order to classify different pattern. In text classification technique, term weighting methods design suitable weights to the specific terms to enhance the text classification performance. This paper surveys of text classification, process of different term weighing methods and comparison between different classification techniques.

Long-term prediction model of rockburst in underground openings using heuristic algorithms and support vector machines

Using information technology such as search engine for collecting and monitoring of network public opinion is a practical and effective method. This paper puts forward an improved algorithm of SVM classifier based on incremental learning online, and then implements a topic crawler system for network public opinion and to grab the public opinion.

In this paper, we propose genetic algorithm (GA) to improve support vector machines (SVM) based intrusion detection system (IDS). SVM is relatively a novel classification technique and has shown higher performance than traditional learning methods in many applications. So several security researchers have proposed SVM based IDS. We use fusions of GA and SVM to enhance the overall performance of SVM based IDS. Through fusions of GA and SVM, the optimal detection model SVM classifier can be determined. As the result of this fusion, SVM based IDS not only select optimal parameters for SVM but also optimal feature set among the whole feature set. We demonstrate the feasibility of our method by performing several experiments on KDD 1999 intrusion detection system competition dataset.

Sentiment Analysis is a widely used text classification technique. It breaks down any given text or comments and classify the text either as positive or negative based on the views conveyed in it. Previous works done on sentiment classification used either lexicon based approach or machine learning techniques. Likewise, the major drawback of the existing systems was the focus on only binary classification of review such as positive or negative. Ignorance of the neutral review will result in misinterpretation of a customer's opinion about a product or movie, which will degrade the business or trend. In case of using only lexicon based approach, the system highly depends on the selection of lexicon resource and dictionary. In case of system built only using machine learning approach, the performance of the system depends on the algorithms chosen. This work presents a hybrid model to resolve the neutral class too. The proposed work combines a lexical approach (SentiWordNet) with the machine learning algorithms such as Support Vector Machine, Decision Tree, Logistic Regression and Naive Bayes for sentiment analysis to resolve the neutral opinions beyond the binary categorization of the customer's review. We have also compared the performance of these four machine learning algorithms along with the lexicon approach. The results proved that Support Vector Machine and Logistic Regression algorithms outperform the other two algorithms with an accuracy of about 80% which is on average differs by 6% to 10% when compared to other algorithms.

Due to errors, accuracy of Global Positioning System is not so high. Therefore, the Real Time Differential Global Poisoning System (RTDGPS) which is based on the successive transmission message of RTCM protocol, is using in real-time applications. Stability and accuracy of the system, significantly depends to a fast transmission of correction messages. These messages come from the reference station to the user stations and affected by the errors related to each satellite. Receiving correction factors which are transmitted by the reference station are facing with time-lag problem, which can increase the error of the RTDGPS. To overcome this problem, prediction algorithms are used. In this research, support vector machine (SVM) model is used to predict the pseudo range correction. Unfortunately, the practical use of SVM is limited because the quality of SVM models depends on a proper setting of SVM and SVM kernel parameters. Therefore, to determine the main parameters of the SVM, both particle swarm optimization (PSO) and genetic algorithm (GA), as two optimization techniques, are used. The proposed methodology has been implemented by a 6-s predicts time step. Simulations showed that the accuracy of GA–SVM and PSO–SVM are equal to 0.186 and 0.154, respectively.

흡수, 분포, 대사, 배설 특성 및 독성을 예측하기 위한 효과적인 툴을 개발하는 것은 신약개발의 초기단계에서 NCE(new chemical entity)에 대한 가장 중요한 업무 중의 하나이다. 최근에 이런 시도중의 하나로서 ADME/T(absorption, distribution, metabolism, excretion, toxicity)관련 성질들의 예측에 support vector machine(SVM)을 이용하고 있다. 그리고 SVM은 ADME/T 성질들을 정확하게 예측하는데 많이 사용 되고 있다. 그러나 SVM 모델링에 두 가지 문제가 있다. 특성 선택(feature selection) 과 매개변수 설정(parameter setting)은 여전히 해결해야 할 과제이다. 이 두 가지 문제들은 SVM 분류의 효율성과 정확도에 결정적인 영향을 끼친다. 특히 특성 선택과 최적화된 SVM 변수의 설정은 서로 영향을 주기 때문에 동시에 다루어져야 한다. 여기서 우리는 genetic algorithm(GA) – 특성 선택에 사용 – 과 grid search(GS) method– 변수최적화에 사용 – 두 가지를 통합하는 효과적인 해결책을 제시하였다. ADME/T관련 성질 중 하나인 심장부정맥을 야기시키는 hERG 이온채널 저해제 분류 모델이 여기서 제안된 GA-GS-SVM을 위해 할당되고 테스트 되었다. 1891개의 화합물을 가지는 트레이닝 셋으로 단일 모델 3개, 앙상블 모델 3개, 총 6개의 모델을 만들었고 175개의 외부 데이터를 테스트 셋으로 사용하여 검증하였다. 데이터의 불균형 문제를 해결하기 위하여 GA-GS-SVM 단일 모델에 의한 예측 정확도와 GA-GS-SVM 앙상블 모델 예측 정확도를 비교하였으며, 앙상블모델을 사용하여 예측의 정확도를 높일 수 있었다. Developing effective tools for predicting absorption, distribution, metabolism, excretion properties and toxicity (ADME/T) of new chemical entities in the early stage of drug design is one of the most important tasks in drug discovery and development today. As one of these attempts, support vector machines (SVM) has recently been exploited for the prediction of ADME/T related properties. However, two problems in SVM modeling, i.e. feature selection and parameters setting, are still far from solved. The two problems have been shown to be crucial to the efficiency and accuracy of SVM classification. In particular, the feature selection and optimal SVM parameters setting influence each other, which indicates that they should be dealt with simultaneously. In this account, we present an integrated practical solution, in which genetic-based algorithm (GA) is used for feature selection and grid search (GS) method for parameters optimization. hERG ion-channel inhibitor classification models of ADME/T related properties has been built for assessing and testing the proposed GA-GS-SVM. We generated 6 different models that are 3 different single models and 3 different ensemble models using training set - 1891 compounds and validated with external test set - 175 compounds. We compared single model with ensemble model to solve data imbalance problems. It was able to improve accuracy of prediction to use ensemble model.

This paper addresses processing of spectrophotometric array signals based on genetic algorithms (GA) least square support vector machines (LS-SVM) regression to provide a powerful model for machine learning and data mining. The key to complete LS-SVM regression is to choose its optimal parameters. Due to their outstanding ability in solving global optimization problems in complex multidimensional search space, GA are used in this study to obtain the optimal parameter combination of the LS-SVM model. Experimental results showed the GA-LS-SVM method to be successful for simultaneous multicomponent determination even where severe overlap of spectra was present. KeywordsLeast squares support vector machines; Genetic algorithms; Spectrophotometric array signals; Overlapping spectra; Artificial intelligence Nowadays, with the application of photometric diode array detector and computers, rapid scanning commercial spectrophotometers are capable of quickly generating huge data consisting of hundreds and even thousands of absorbance values per spectrum. The array data named fullspectrum contain sufficient information to be able to determine the contents of various compounds. The main drawback of ultraviolet-visible (UV-VIS) is its poor selectivity because in many cases UV-VIS spectra display strong overlaps, especially some less specific and selective chromagenic reagents often give rise to strongly overlapped spectra in many cases. The combination of artificial intelligence methods with the computer-controlled spectrophotometers was proven to be effective in overcoming this difficulty [1-3]. Artificial neural network (ANN) is a form of artificial intelligence that mathematically simulates biological nervous system [4, 5]. However, ANN often has slow convergence, is prone to the existence of many local minima during training, and has a tendency of overfitting. Recently, a promising technology called support vector machines (SVM) has been used for classification and regression problems. SVM pioneered by Vapnik is a kind of machine learning method based on modern statistical learning theory and has notable properties including absence of local minima and high generalization ability [6, 7]. Suykens and his coworkers [8] introduced a modified version of SVM called least square SVM (LS-SVM) , which requires solving a set of linear equations instead of a quadratic programming problem and is much easier and computationally simpler than SVM. SVM and LS-SVM represent relatively recent artificial intelligence method and have found some applications in image analysis, classification and disease diagnosis etc. [9, 10]. It is worth mentioning that the success of LS-SVM model is highly dependent on the optimum choice of two parameters, the relative weight of regression error γ and the kernel width σ of radial basis function (RBF). Genetic algorithms (GA) [11, 12] introduced by John Holland are probabilistic optimization techniques based on natural evolution and genetics and Darwin’s theory of survival of the best. With their efficient and robust global search ability, GA are used to search two optimal parameters for the LS-SVM model simultaneously and automatically. The LS-SVM model then performs the regression task using these optimal parameters.

A massive of text that is generated every minute is increasing dramatically. Therefore, it is more and more important to find an effective model to automatic classify the amount of text. Topic models is the most powerful techniques in text classification. There are many research results in the field of topic model have been published in scholarly journals. The Latent Dirichlet Allocation (LDA) is one of the most popular topic models in text classification. Researchers have proposed many topic evolution models based on LDA to solve some specific problems in applications of text classification. And some joint models which based on topic models combined other algorithms have been studied to enhance the performance of text classification. In this paper, we investigated three categories topic models for text classification and briefly introduced their advantages and disadvantages in the applications of text mining. Also, we introduce the generated process of documents and illustrate the graphical model for each topic models.

Metaheuristic algorithms, inspired by natural phenomena and human-based strategies, offer versatile approaches to navigate diverse search spaces and adapt to dynamic environments. These algorithms, including evolutionary algorithms, swarm intelligence, bio-inspired methods, human-based approaches, and plant-inspired techniques, have found applications across diverse domains such as engineering, finance, healthcare, logistics, and telecommunications. In the text classification domain, metaheuristic techniques have emerged as powerful tools to enhance the accuracy, efficiency, and robustness of classification systems. By optimizing feature subsets, fine-tuning model parameters, and addressing challenges such as feature selection, dimensionality reduction, class imbalance, and noisy data, metaheuristic algorithms provide flexible solutions that adapt to various text datasets and tasks. This review paper comprehensively explores recent advancements in metaheuristic applications in text classification across six categories. From evolutionary-based methods to swarm-based approaches, bio-inspired techniques to physics/chemistry-based strategies, human-based methods to plant-based algorithms, researchers have leveraged diverse metaheuristic techniques to push the boundaries of text classification. Through a systematic analysis of recent research studies, this review provides insights into the strengths, limitations, and future directions of metaheuristic optimization in the context of text classification.