A novel text classification based on Mahalanobis distance

Machine learning classifiers vary concerning their learning functions and approach. K–nearest neighbour (K-NN) is a versatile instance-based machine learning algorithm used for classification without building a learning model. Varying distance measures in K-NN for computing distance between instances affect the classification accuracy. Commonly used distance metrics are Euclidean, Manhattan, Minkowski and Mahalanobis. In this work, extensive experiments were carried out on diverse datasets. These datasets vary in terms of the domain, number and types of features. The classification accuracy results obtained by applying different distance measures in the K-NN classifier were compared and analyzed. The results show that the Mahalanobis distance performed better with more than 90% accuracy, than the Euclidean and other measures when applied on most datasets. The other two most significant inferences from this analysis are: first, the Mahalanobis distance considered the correlation between the features and the deviation of instances from the mean, and second, it is scale-invariant.

Traditional k-nearest neighbor methods couldn’t be able to correctly classify objects when their k nearest neighbors are dominated by other classes. This paper formulates a two-class classification problem, and applies a modified k-nearest neighbors (KNN) classifier algorithm based on maximal coherence, validity ratings, and k-fold cross validation to classify the test samples. We build a validity score for the pairs of sample and their surroundings according to their labels. The k nearest neighbors (including the unknown test object) of each sample in the training set as well as the unknown test object itself will be determined. The unknown test object will be tentatively assigned to a class membership. Then we use the validity scores to quantify the degree to which a pre-determined group of samples resemble their k nearest neighbors. A classifier is designed which take into account the coherence and validity ratings. A numerical example demonstrates the effectiveness of the algorithm in detail. The enhanced KNN method is compared with the conventional KNN and the modified KNN method on both real world wine data and photo-thermal infrared imaging spectroscopy (PT-IRIS) data for up to 20 different k values. Classification accuracy of KNN method and our method in terms of various combinations of k-value and k-fold cross validation are compared. The experimental results show that the proposed enhanced KNN method outperforms the conventional KNN and the modified KNN method on both real world wine data and PT-IRIS data. In addition, the classification accuracy of both the conventional KNN and our method increase drastically when k = 5. The average classification accuracy of our method on the PT-IRIS data featuring small sample size and high overlap is 97.87%.

Semantic text classification: A survey of past and recent advances

The field of Text Mining has evolved over the past years to analyze textual resources. However, it can be used in several other applications. In this research, we are particularly interested in performing text mining techniques on audio materials after translating them into texts in order to detect the speakers' emotions. We describe our overall methodology and present our experimental results. In particular, we focus on the different features selection and classification methods used. Our results show interesting conclusions opening up new horizons in the field, and suggest an emergence of promising future work yet to be discovered.

In the text classification, The similarity between the text need to be calculated, but the existing classification methods only consider the similarity between feature words and categories and does not involve the semantic similarity between feature words. In this paper, a new classification model LDA (Latent Dirichlet Allocation) - KNN (K-Nearest Neighbor) is proposed. LDA is used to solve the problem of semantic similarity measurement in traditional text categorization. The sample space is modeled and selected by this model. In the reduced feature space, KNN classifier is used to classify the sample. The experiment was based on the Matlab software platform, and the data set was obtained from the Chinese corpus of Fudan University, and the high precision classification result was obtained with the average value of 0.933. LDA-KNN model is compared with MI(Mutual Information)-KNN model and LSI(Latent Semantic Index)-KNN model. The results show that LDA-KNN model has superior classification performance in automatic text categorization.

The k Nearest Neighbors (KNN) algorithm has been widely applied in various supervised learning tasks due to its simplicity and effectiveness. However, the quality of KNN decision making is directly affected by the quality of the neighborhoods in the modeling space. Efforts have been made to map data to a better feature space either implicitly with kernel functions, or explicitly through learning linear or nonlinear transformations. However, all these methods use pre-determined distance or similarity functions, which may limit their learning capacity. In this paper, we propose a novel deep learning architecture, which is called the Deep Similarity-Enhanced K Nearest Neighbors (DSE-KNN), to learn an optimized similarity function of the data directly towards the goal of optimizing the KNN decision making. In other words, the type of similarity function that is used in our method is not pre-determined but rather learned to map data to a high-dimensional feature space where the accuracy of the KNN decision making is maximized. Experimental results show that DSE-KNN outperforms other common machine learning methods on classifying different types of disease datasets and predicting daily price direction of different stock ETFs.

Text mining is a special case of data mining which explore unstructured or semi-structured text documents, to establish valuable patterns and rules that indicate trends and significant features about specific topics. Text mining has been in pattern recognition, predictive studies, sentiment analysis and statistical theories in many areas of research, medicine, financial analysis, social life analysis, and business intelligence. Text mining uses concept of natural language processing and machine learning. Machine learning algorithms have been used and reported to give great results, but their performance of machine learning algorithms is affected by factors such as dataset domain, number of classes, length of the corpus, and feature selection techniques used. Redundant attribute affects the performance of the classification algorithm, but this can be reduced by using different feature selection techniques and dimensionality reduction techniques. Feature selection is a data preprocessing step that chooses a subset of input variable while eliminating features with little or no predictive information. Feature selection techniques are Information gain, Term Frequency, Term Frequency-Inverse document frequency, Mutual Information, and Chi-Square, which can use a filters, wrappers, or embedded approaches. To get the most value from machine learning, pairing the best algorithms with the right tools and processes is necessary. Little research has been done on the effect of feature selection techniques on classification accuracy for pairing of these algorithms with the best feature selection techniques for optimal results. In this research, a text classification experiment was conducted using incident management dataset, where incidents were classified into their resolver groups. Support vector machine (SVM), K-Nearest Neighbors (KNN), Naïve Bayes (NB) and Decision tree (DT) machine learning algorithms were examined. Filtering approach was used on the feature selection techniques, with different ranking indices applied for optimal feature set and classification accuracy results analyzed. The classification accuracy results obtained using TF were, 88% for SVM, 70% for NB, 79% for Decision tree, and KNN had 55%, while Boolean registered 90%, 83%, 82% and 75%, for SVM, NB, DT, and KNN respectively. TF-IDF, had 91%, 83%, 76%, and 56% for SVM, NB, DT, and KNN respectively. The results showed that algorithm performance is affected by feature selection technique applied. SVM performed best, followed by DT, KNN and finally NB. In conclusion, presence of noisy data leads to poor learning performance and increases the computational time. The classifiers performed differently depending on the feature selection technique applied. For optimal results, the classifier that performed best together with the feature selection technique with the best feature subset should be applied for all types of data for accurate classification performance. Keywords: Text Classification, Supervised Machine Learning, Feature Selection DOI : 10.7176/JIEA/9-3-06 Publication date :May 31 st 2019

The availability of massive sets of unstructured data has opened up new opportunities for businesses to gain meaningful insights into what is currently happening in their organization and market place. Unfortunately, unstructured data is often challenging to work with, as it requires specialized toolsets, techniques, knowledge and skills to engage with the data. This study used a systematic review (SLR) process to explore the current and conversant state of affairs in the field of text mining (TM) techniques that exist for the processing of natural language in unstructured text datasets across a broad expanse of applications and domains. A comprehensive literature search yielded 1022 eligible articles from five prominent bibliographic databases, narrowed down to 89 articles for review. Information related to TM techniques, the TM process, applications, challenges and recommendations for the improvement of TM results were extracted and synthesized. Eighteen TM techniques were identified and used to complete a variety of tasks such as data pre-processing, information retrieval, information extraction, text classification, clustering, topic modeling, text summarization and sentiment analysis or opinion mining. No single TM technique may be suitable for all text representation and extraction requirements. Therefore, context and applicability are key factors for TM technique selection.KeywordsText miningText mining techniquesUnstructured dataKnowledge discovery

Probability theory is the framework for making decision under uncertainty. In classification, Bayes' rule is used to calculate the probabilities of the classes and it is a big issue how to classify raw data rationally to minimize expected risk. Bayesian theory can roughly be boiled down to one principle: to see the future, one must look at the past. Naive Bayes classifier is one of the mostly used practical Bayesian learning methods. K-nearest neighbor is a supervised learning algorithm where the result of new instance query is classified based on majority of k-nearest neighbor category. The classifiers do not use any model to fit and only based on memory/training data. In this paper, after reviewing Bayesian theory, the naive Bayes classifier and k-nearest neighbor classifier is implemented and applied to a dataset "credit card approval" application. Eventually the performance of these two classifiers is observed on this application in terms of the correct classification and misclassification and how the performance of k-nearest neighbor classifier can be improved by varying the value of k.

The majority of the data is preserved as text (about 75%), hence It is believed that text mining has a significant commercial potential. Unstructured texts continue to be the most readily available source of knowledge, despite the fact that knowledge may be accessed in many other places. Text classification that assigns documents to predetermined categories. Machine learning approaches can categories texts more accurately. The goal of this work is to introduce text classification, give a description of the text classification technique, a general review of the classifiers, and a comparison of a few of the current classifiers. It is based on performance, time complexity, and other factors. On the basis of speed, accuracy, benefits, and drawbacks of existing classification methods such as Decision Tree, Naive Bayes, Support Vector Machine, and k-Nearest Neighbours are compared.

FSKNN: Multi-label text categorization based on fuzzy similarity and k nearest neighbors

UWB radar with high-range resolution and strong penetration ability can be used to separate multiple human targets in a complex environment. The through-wall human being detection with UWB radar has been relatively mature in the current study. This paper extracts the characteristic parameters which are related to the human targets from the received signals as the sample data. And used machine learning based on the KNN (K nearest neighbor) classification algorithm to identify and classify the through-wall human being status. Experimental results showed that the KNN classification algorithm effectively distinguished three statuses of through-wall human being and reached the prospective goal.

Mainly, the holy Quran is the holy book for all Muslims. Reading the holy Quran is a special reading with rules. Reading the Holy Quran is called recitation. One of the Muslim essential activities is reading or listening to the Holy Quran. In this paper, a machine learning approach for recognizing the reader of the holy Quran (reciter) is proposed. The proposed system contains basic traditional phases for a recognition system, including data acquisition, pre-processing, feature extraction, and classification. A dataset is created for ten well-known reciters. The reciters are the prayer leaders in the holy mosques in Mecca and Madinah. The audio dataset set is analyzed using the Mel Frequency Cepstral Coefficients (MFCC). Both the K nearest neighbor (KNN) classifier, and the artificial neural network (ANN) classifier are applied for classification purpose. The pitch is used as features which are utilized to train the ANN and the KNN for classification. Two chapters from the Holy Quran are selected in this paper for system validation. Excellent accuracy is achieved. Using the ANN, the proposed system gives 97.62% accuracy for chapter 18 and 96.7% accuracy for chapter 36. On the other hand, the proposed system gives 97.03% accuracy for chapter 18 and 96.08% accuracy for chapter 36 by using the KNN.

Aim: Coronary illness is one of the main sources of mortality in this current world. A basic test in the field of clinical information assessment is the expectation of cardiovascular infection. The fundamental target of this paper is to work on the exactness of the Data mining algorithms being used to improve heart disease prediction. We will be comparing the novel Random forest with K Nearest Neighbor to find out which of these can give us the best accuracy. Material and Methods: The study used 98 samples with Novel Random Forest and K Nearest Neighbor is executed with varying training and testing splits for predicting the accuracy for heart disease prediction with the 80% of G-power value and data about cardiovascular disease was gathered from numerous web sources, including latest research findings and threshold 0.05%, confidence interval 95% mean and standard deviation. The performance of the accuracy rate of the classifiers is examined using the heart disease dataset.. There was a statistical insignificant difference among Innovative Random Forest and K Nearest Neighbor p = 0.484 (p>0.05). Results and Discussion: The accuracy of predicting heart disease in Novel Random Forest 90.16% and K Nearest Neighbor 67.21% is obtained. Conclusion: Prediction of Heart disease using the innovative Random Forest (RF) technique appears to be a significant improvement over the K Nearest Neighbor (KNN) algorithm in terms of accuracy.

Abstract- This study uses a K-Nearest Neighbours (KNN) classifier to detect teenage cardiac disease. Years of age, Sexual orientation, Chest Ache category, Ambient Heart pressure, cholesterol levels, Highest Cardiac Level, and Exercise-Induced Angina are clinically important and interpretable aspects of the cardiovascular disease dataset used to create the model. These qualities were chosen to improve model interpretability for doctors and laypeople. Data preparation encoded categorical variables and standardised features for model optimisation. Histograms, charts with bars, and scatter plots were used to explore feature distribution and heart disease status. The kernel nearest neighbour (KNN) model was trained with k=11 neighbours and tested using precision, a matrix of confusion, classifying report, ROC spectrum, and precision-recall curve, proving predictive power. To select the most influential predictors, feature importance was permutation-based. A Flask web application lets users input health parameters to forecast heart disease risk. The software generates personalised, patient-friendly health information using OpenAI's API and generates PDF reports. The technology keeps prognosis records in an information system for future reference. This approach uses data-driven modelling, interpretability, and user-friendliness to help adolescents recognise and understand heart disease risk. Keywords - Adolescent heart disease identification, risk factors for heart disease, KNN classifier, artificial intelligence, data preprocessing, feature relevance, model assessment. Flask web app, OpenAI integration into the API, health insights, PDF reports, early diagnosis, and healthcare information visualisation.