Dissimilarity based feature selection for text classification

In this paper, a simple and efficient symbolic text classification is presented. A text document is represented by the use of interval valued symbolic features. Subsequently, a new feature selection method based on a new dissimilarity measure is also presented. The new feature selection method reduces the features in the representation phase for effective text classification. It keeps the best features for effective text representation and simultaneously reduces the time taken to classify a given document. To corroborate the efficacy of the proposed method, experimentation has been conducted on four different datasets to evaluate the performance. Experimental results reveal that the proposed method gives better results when compare to state of the art techniques. In addition, as it is based on simple matching scheme it achieves classification within negligible time and thus it appear to be more effective in classification.

Feature selection is an important task in machine learning. It can improve classification accuracy and effectively reduce the dataset dimensionality by removing no discriminative features. Though a large body of researches were focused on feature selection for text classification, few works addressed the problem for multi-label data in big data context. Therefore, this paper proposes a distributed feature selection approach for multi-label textual big data based on the weighted chi-square method. First, a standard multi-label approach to transform the multi-label data into single-label data is applied. Then, the algorithm assigns different weights to the features based on the category term frequency and then calculates the chi-square based on the weight of each feature. The proposed method is implemented on Hadoop framework using MapReduce programming model. At last, a set of experiments were conducted on three benchmarking text datasets to evaluate the effectiveness of the proposed approach. A comparative analysis of the results with the state-of-the-art techniques proves that our method is efficient, robust and scalable.

The use of ensemble paradigm with classifiers is a proven approach that involves combining the outcomes of several classifiers. It has recently been extrapolated to feature selection methods to find the most relevant features. Earlier, ensemble feature selection has been used in high dimensional, low sample size datasets like bioinformatics. To one's knowledge there is no such endeavor in the text classification domain. In this work, the ensemble feature selection using data perturbation in the text classification domain has been used with an aim to enhance predictability and stability. This approach involves application of the same feature selector to different perturbed versions of training data, obtaining different ranks for a feature. Previous works focus only on one of the metrics, that is, stability or accuracy. In this work, a combined framework is adopted that assesses both the predictability and stability of the feature selection method by using feature selection ensemble. This approach has been explored on univariate and multivariate feature selectors, using two rank aggregators.

A well representing feature set that has enough differentiating power plays an important role in classification. The existing techniques for feature set selection are mostly statistical. They only make decision between a feature inclusion or exclusion without taking into account the importance of human thinking and reasoning that surely effect the results of classification and prediction. In this paper, I propose a new method of feature selection and employed it via Fuzzy Set Theory (FST). The effectiveness of proposed method is evaluated in terms of improving the classifier's performance as compared to state of the art methods such as Gini Index, Information Gain, and Document Frequency. A case study is performed with widely known classifiers Naive Bayes, Support Vector Machine and Random Forest in the context of classification of software design patterns. The promising results indicate the applicability of proposed method in context of automated systems of certain domains.

This paper presents an improved Gini-Index algorithm to correct feature-selection bias in text classification. Gini-Index has been used as a split measure for choosing the most appropriate splitting attribute in decision tree. Recently, an improved Gini-Index algorithm for feature selection, designed for text categorization and based on Gini-Index theory, was introduced, and it has proved to be better than the other methods. However, we found that the Gini-Index still shows a feature selection bias in text classification, specifically for unbalanced datasets having a huge number of features. The feature selection bias of the Gini-Index in feature selection is shown in three ways: 1) the Gini values of low-frequency features are low (on purity measure) overall, irrespective of the distribution of features among classes, 2) for high-frequency features, the Gini values are always relatively high and 3) for specific features belonging to large classes, the Gini values are relatively lower than those belonging to small classes. Therefore, to correct that bias and improve feature selection in text classification using Gini-Index, we propose an improved Gini-Index (I-GI) algorithm with three reformulated Gini-Index expressions. In the present study, we used global dimensionality reduction (DR) and local DR to measure the goodness of features in feature selections. In experimental results for the I-GI algorithm, we obtained unbiased feature values and eliminated many irrelevant general features while retaining many specific features. Furthermore, we could improve the overall classification performances when we used the local DR method. The total averages of the classification performance were increased by 19.4%, 15.9%, 3.3%, 2.8% and 2.9% (kNN) in Micro-F1, 14%, 9.8%, 9.2%, 3.5% and 4.3% (SVM) in Micro-F1, 20%, 16.9%, 2.8%, 3.6% and 3.1% (kNN) in Macro-F1, 16.3%, 14%, 7.1%, 4.4%, 6.3% (SVM) in Macro-F1, compared with tf*idf, χ2, Information Gain, Odds Ratio and the existing Gini-Index methods according to each classifier.

In text classification, feature selection is essential to improve the classification effectiveness. This paper provides an empirical study of a feature selection method based on genetic algorithms for different text representation methods. This feature selection algorithm can accomplish two goals: in one hand is the search of a feature subset such that the performance of classifier is best; in other hands is find a feature subset with the smallest dimensionality which achieves higher accuracy in classification. To evaluate the performance of this approach, three from the best classifiers have been selected: Naive Bayes (NB), Nearest Neighbors (KNN) and Support Vector Machines (SVMs). Our objective is to determine whether the genetic algorithms based feature selection will improve the performances in text classification with smaller size using F-measure. Experimentations were carried out on two benchmark document collections 20Newsgroups, and Reuters-21578. And the results were very interesting.

Many feature selection methods have been proposed for text categorization. However, their performances are usually verified by experiments, so the results rely on the corpora used and may not be accurate. This paper proposes a novel feature selection framework called Distribution-Based Feature Selection (DBFS) based on distribution difference of features. This framework generalizes most of the state-of-the-art feature selection methods including OCFS, MI, ECE, IG, CHI and OR. The performances of many feature selection methods can be estimated by theoretical analysis using components of this framework. Besides, DBFS sheds light on the merits and drawbacks of many existing feature selection methods. In addition, this framework helps to select suitable feature selection methods for specific domains. Moreover, a weighted model based on DBFS is given so that suitable feature selection methods for unbalanced datasets can be derived. The experimental results show that they are more effective than CHI, IG and OCFS on both balanced and unbalanced datasets.

Big multimedia data is heterogeneous in essence, that is, the data may be a mixture of video, audio, text, and images. This is due to the prevalence of novel applications in recent years, such as social media, video sharing, and location based services (LBS), etc. In many multimedia applications, for example, video/image tagging and multimedia recommendation, text classification techniques have been used extensively to facilitate multimedia data processing. In this paper, we give a comprehensive review on feature selection techniques for text classification. We begin by introducing some popular representation schemes for documents, and similarity measures used in text classification. Then, we review the most popular text classifiers, including Nearest Neighbor (NN) method, Naive Bayes (NB), Support Vector Machine (SVM), Decision Tree (DT), and Neural Networks. Next, we survey four feature selection models, namely the filter, wrapper, embedded and hybrid, discussing pros and cons of the state-of-the-art feature selection approaches. Finally, we conclude the paper and give a brief introduction to some interesting feature selection work that does not belong to the four models.

We consider feature selection for text classification both theoretically and empirically. Our main result is an unsupervised feature selection strategy for which we give worst-case theoretical guarantees on the generalization power of the resultant classification function f with respect to the classification function f obtained when keeping all the features. To the best of our knowledge, this is the first feature selection method with such guarantees. In addition, the analysis leads to insights as to when and why this feature selection strategy will perform well in practice. We then use the TechTC-100, 20-Newsgroups, and Reuters-RCV2 data sets to evaluate empirically the performance of this and two simpler but related feature selection strategies against two commonly-used strategies. Our empirical evaluation shows that the strategy with provable performance guarantees performs well in comparison with other commonly-used feature selection strategies. In addition, it performs better on certain datasets under very aggressive feature selection.

Many studies in automated Text Categorization focus on the performance of classifiers, with or without considering feature selection methods, but almost as a rule taking into account just one document representation. Only relatively recently did detailed studies on the impact of various document representations step into the spotlight, showing that there may be statistically significant differences in classifier performance even among variations of the classical bag-of-words model. This paper examines the relationship between the idf transform and several widely used feature selection methods, in the context of Naive Bayes and Support Vector Machines classifiers, on datasets extracted from the dmoz ontology of Web-page descriptions. The described experimental study shows that the idf transform considerably effects the distribution of classification performance over feature selection reduction rates, and offers an evaluation method which permits the discovery of relationships between different document representations and feature selection methods which is independent of absolute differences in classification performance.

In the past research, Document Frequency (DF) has been validated to be a simple yet quite effective measure for feature selection in text classification. The calculation is based on how many documents in a collection contain a feature, which can be a word, a phrase, a n-gram, or a specially derived attribute. The counting process takes a binary strategy: if a feature appears in a document, its DF will be increased by one. This traditional DF metric concerns only about whether a feature appears in a document, but does not consider how important the feature is in that document. Obviously, thus counted document frequency is very likely to introduce much noise. Therefore, a weighted document frequency (WDF) is proposed and expected to reduce such noise to some extent. Extensive experiments on two text classification datasets demonstrate the effectiveness of the proposed measure.

Weighting formula and feature selection are key preprocessing in text classifying and mining. We analyze the drawbacks of weighting formula based on inverse document frequency and present a novel feature weighting and selecting method based on variable precision rough set model. Inverse document frequency (IDF) doesn't take the classification information into account and the criterion based on IDF is not monotonous with the contribution that a feature makes to classification, which decreases the classifier's performance. The measure of classification quality based on variable rough set model can deal with complex classification. It measures the contribution a feature makes to classification. It is introduced as a criterion for feature selecting and weighting in text classification. We name it as TFACQ. The experimental results show that the weighting formula and feature selection based on TFACQ have greatly improved the performance.

Feature Selection (FS) is the selection of certain features by using a criterion assessment application. In the age of technological advancements and innovations, there has been a significant rise in the use of different social networking applications such as Facebook, YouTube, Twitter, and e-commerce platforms such as Amazon, Alibaba, Flipkart, and others. It has increased the use of sentiment analysis in the different real-world applications so that there is a classification of text in different preset polarities such as optimistic, pessimistic, and impartial. The current research aims to find out the selection process involved in choosing the optimal features for classification using sentimental textual data. The study presents the scope of the sentimental analysis using textual data, fundamentals of the classification process, and the significance of the feature selection schemes so that there is an implementation of effective feature selection for text classification and analysis of sentiments.

Due to the existence of text data, multi-label (ML) text classification is an essential task in machine learning. Feature selection is an essential and effective preprocess to enhance the learning process. Choosing a Multi-Label Feature Selection (MLFS) algorithm is the most basic, critical, and sensitive choice in ML classification operations. If this choice is based on a criterion, it cannot be attributed to always being sound. Choosing the best algorithm must be evaluated using several different criteria to be examined from different aspects. In this article, we turn the issue into an election and use the Weighted Borda Count method for voting. We do the voting in three stages continuously so that a subset of different features does the voting. In the second stage, voting of different methods is done with six criteria, and each criterion selects the methods in order of priority from the beginning to the end. Voting steps 1 and 2 are performed on eighteen text datasets used. Finally, in the final voting stage, the methods are evaluated and voted on by different text datasets. The final result of the voting in the third stage shows the desired MLFS methods based on their performance from beginning to end. According to the experiments performed and the results obtained, it can be seen that the selection of the algorithm based on several different criteria and considering the overall performance of the algorithm will be better than the selection based on one criterion.

In this research we apply several machine learning methods and word embedding features to process social media data, specifically comments on the Disney Plus Hotstar application. The word embedding features used include Word2Vec, GloVe, and FastText. Our aim is to evaluate the impact of these features on the classification performance of machine learning methods such as Naive Bayes (NB), K-Nearest Neighbor (KNN), and Random Forest (RF). NB is very simple and efficient and very sensitive to feature selection. Meanwhile, KNN is known for its weaknesses such as biased k values, overly complex computations, memory limitations, and ignoring irrelevant attributes. Then RF has a weakness, namely that the evaluation value can change significantly with just a slight change in the data. Feature selection in text classification is crucial for enhancing scalability, efficiency, and accuracy. Our testing results indicate that KNN achieved the highest accuracy both before and after feature selection. The FastText feature led to the highest performance for KNN, yielding balanced accuracy, precision, recall, and F1-score values.