Efficient Jamming Identification in Wireless Communication: Using Small Sample Data Driven Naive Bayes Classifier

Mining data over streams in one pass and using constant memory is a challenging task. Decision trees are one of the most popular classifiers for both batch and incremental learning due to their high degree of interpretability, ease of construction and good accuracy. The most popular decision tree for stream classification is Hoeffding Tree based on Hoeffding bound. Literature shows a few variants of decision trees based on different bounds. The default class prediction method adopted in decision tree is “majority class” approach. Later, the accuracy of prediction was scaled up by a hybrid decision tree where Naive Bayes classifier was used for prediction. Kernel Density Estimation (KDE) is employed in Flexible Naive Bayes for classification. However, it is suitable for modeling static data set. This paper proposes an Incremental Flexible Naive Bayes (IFNB) based hybrid decision tree paradigm that uses KDE to model continuous attributes at leaf nodes of the tree for improving the class prediction accuracy. Experimental results on both synthetic and real dataset show that the proposed IFNB based leaf classifiers achieves improvement over the class prediction methods adopted in existing decision trees for data streams.

Opinion mining has become a major part in today's economy. People would want to know more about a product and the customers opinion before buying it. Companies would also want to know the opinions of the customers. Therefore, analyzing the customer’s opinion is important. A new customer would consider a product as good by analyzing the opinions of other customers. The opinions are collected from various areas, which include blogs, web forums, and product review sites. Classifying these large set of opinions requires a good classifier. In view of this, a comparative study of three classification techniques - Naive Bayes classifier with Kernel Density Estimation (KDE), Support Vector Machine (SVM), Decision Tree and KNN was made. To evaluate the classifier accuracy, precision, recall and F-measure techniques are used. Experimental results show that the Naive Bayes with Kernel Density Estimation (KDE) classifier achieved higher accuracy among others.

Kernel density estimation (KDE) is an important method in nonparametric learning. While KDE has been studied extensively in the context of accuracy of distribution estimation, it has not been studied extensively in the context of classification. This paper studies nine bandwidth selection schemes for kernel density estimation in Naive Bayesian classification context, using 52 machine learning benchmark datasets. The contributions of this paper are threefold. First, it shows that some commonly used and very sophisticated bandwidth selection schemes do not give good performance in Naive Bayes. Surprisingly, some very simple bandwidth selection schemes give statistically significantly better performance. Second, it shows that kernel density estimation can achieve statistically significantly better classification performance than a commonly used discretization method in Naive Bayes, but only when appropriate bandwidth selection schemes are applied. Third, this study gives bandwidth distribution patterns for the investigated bandwidth selection schemes.

Naive Bayes (NB) classifier is a simple and efficient classifier, but the independent assumption of its attribute limits the application of the actual data. This paper presents an approach called particle swarm optimization-naive Bayes (PSO-NB) which takes advantage of combination particle swarm optimization with naive Bayes for attribute selection to improve naive Bayes classifier. This method applies PSO firstly to search out an optimal subset of attributes reduction in the original attribute space, and then constructs a naive Bayes classifier on the gotten subset of the attributes reduction. Nineteen experimental results on UCI datasets distinctly show that compared with Cfs-BestFirst algorithm, NB algorithm, Decision Tree(C4.5) algorithm, K-neighbor(KNN) algorithm, the proposed algorithm has higher classification accuracy.

Handwritten signature identification is a process that determines an individual’s true identity by analyzing their signature. This is an important task in various applications such as financial transactions, legal document verification, and biometric systems. Various techniques have been developed for signature identification, including feature-based methods and machine learning-based methods. However, verifying handwritten signatures in digital transactions and remote document authentication is still challenging. The inherent variety in people’s signatures, which may occur due to factors such as mood, exhaustion, or even the writing tool used, contributes to the problem. Furthermore, the proliferation of sophisticated forgery methods, such as freehand mimicking and sophisticated picture manipulation, necessitates the development of reliable and precise tools for identifying authentic signatures from fake ones.The present paper suggests a method for identifying signatures based on integrating static (off-line) handwritten signature data. This is done by fusing three types of signature features: Linear Discriminant Analysis (LDA) as appearance-based features, Fast Fourier Transform (FFT) as frequency-features, and Gray-Level Co-occurrence Matrix (GLCM) as texture-features. Then, these fused features are inputted into four types of machine learning algorithms: Naive Bayes, K-Nearest Neighbor, Decision Tree, and AdaBoost classifiers, to identify each person and to find the most robust algorithm in terms of accuracy and precision and recall. For experiments, we have used two famous datasets: SigComp2011 and CEDAR. After training datasets, the highest accuracy achieved was 100% on the CEDAR dataset and 94.43% on the SigComp2011 dataset using a Naive Bayes classifier. Index Terms— Fast Fourier Transform, Gray-Level Co-occurrence Matrix, Handwritten Signature, Linear Discriminant Analysis, Machine Learning.

Text mining is a special case of data mining which explore unstructured or semi-structured text documents, to establish valuable patterns and rules that indicate trends and significant features about specific topics. Text mining has been in pattern recognition, predictive studies, sentiment analysis and statistical theories in many areas of research, medicine, financial analysis, social life analysis, and business intelligence. Text mining uses concept of natural language processing and machine learning. Machine learning algorithms have been used and reported to give great results, but their performance of machine learning algorithms is affected by factors such as dataset domain, number of classes, length of the corpus, and feature selection techniques used. Redundant attribute affects the performance of the classification algorithm, but this can be reduced by using different feature selection techniques and dimensionality reduction techniques. Feature selection is a data preprocessing step that chooses a subset of input variable while eliminating features with little or no predictive information. Feature selection techniques are Information gain, Term Frequency, Term Frequency-Inverse document frequency, Mutual Information, and Chi-Square, which can use a filters, wrappers, or embedded approaches. To get the most value from machine learning, pairing the best algorithms with the right tools and processes is necessary. Little research has been done on the effect of feature selection techniques on classification accuracy for pairing of these algorithms with the best feature selection techniques for optimal results. In this research, a text classification experiment was conducted using incident management dataset, where incidents were classified into their resolver groups. Support vector machine (SVM), K-Nearest Neighbors (KNN), Naïve Bayes (NB) and Decision tree (DT) machine learning algorithms were examined. Filtering approach was used on the feature selection techniques, with different ranking indices applied for optimal feature set and classification accuracy results analyzed. The classification accuracy results obtained using TF were, 88% for SVM, 70% for NB, 79% for Decision tree, and KNN had 55%, while Boolean registered 90%, 83%, 82% and 75%, for SVM, NB, DT, and KNN respectively. TF-IDF, had 91%, 83%, 76%, and 56% for SVM, NB, DT, and KNN respectively. The results showed that algorithm performance is affected by feature selection technique applied. SVM performed best, followed by DT, KNN and finally NB. In conclusion, presence of noisy data leads to poor learning performance and increases the computational time. The classifiers performed differently depending on the feature selection technique applied. For optimal results, the classifier that performed best together with the feature selection technique with the best feature subset should be applied for all types of data for accurate classification performance. Keywords: Text Classification, Supervised Machine Learning, Feature Selection DOI : 10.7176/JIEA/9-3-06 Publication date :May 31 st 2019

Naïve Bayes (NB) Classifier is considered one of the most widely used algorithms to solve classification problems due to its simplicity, high performance and fast computational process, which assumes that feature variables are independent and continuous variables follow gaussian distribution. Due to that most real-life applications may have feature variables that can’t be modeled by gaussian distribution, or may violate the independence assumption, Flexible Naïve Bayes (FNB) Classifier becomes more suitable, which estimates the densities using a non-parametric Kernel Density Estimator (KDE). In this paper, we introduce K-Nearest Neighbor Naïve Bayes (KNNB) Classifier, an improved flexible classifier which uses K-Nearest Neighbor Density Estimator (KNNDE) instead of KDE, such classifier would solve both NB problems, where the independence assumption is removed and the likelihood density is estimated for all feature variables together. Practical applications using R programming language on datasets from different fields were made for both FNB and KNNB classifiers with and without the independence assumption using 10-folds repeated cross validation, despite that FNB achieved greater accuracy than KNNB when dealing with the feature variables separately, the results showed that KNNB classifier performs better than FNB when the independence assumption is removed, meaning that KNNB is more efficient and suitable than FNB in real-life applications, where the independence assumption can be dropped.

Background and aimsThe birth weight of a newborn is a crucial factor that affects their overall health and future well-being. Low birth weight (LBW) is a widespread global issue, which the World Health Organization defines as weighing less than 2,500 g. LBW can have severe negative consequences on an individual’s health, including neonatal mortality and various health concerns throughout their life. To address this problem, this study has been conducted using BDHS 2017–2018 data to uncover important aspects of LBW using a variety of machine learning (ML) approaches and to determine the best feature selection technique and best predictive ML model.MethodsTo pick out the key features, the Boruta algorithm and wrapper method were used. Logistic Regression (LR) used as traditional method and several machine learning classifiers were then used, including, DT (Decision Tree), SVM (Support Vector Machine), NB (Naïve Bayes), RF (Random Forest), XGBoost (eXtreme Gradient Boosting), and AdaBoost (Adaptive Boosting), to determine the best model for predicting LBW. The model’s performance was evaluated based on the specificity, sensitivity, accuracy, F1 score and AUC value.ResultsResult shows, Boruta algorithm identifies eleven significant features including respondent’s age, highest education level, educational attainment, wealth index, age at first birth, weight, height, BMI, age at first sexual intercourse, birth order number, and whether the child is a twin. Incorporating Boruta algorithm’s significant features, the performance of traditional LR and ML methods including DT, SVM, NB, RF, XGBoost, and AB were evaluated where LR, had a specificity, sensitivity, accuracy and F1 score of 0.85, 0.5, 85.15% and 0.915. While the ML methods DT, SVM, NB, RF, XGBoost, and AB model’s respective accuracy values were 85.35%, 85.15%, 84.54%, 81.18%, and 84.41%. Based on the specificity, sensitivity, accuracy, F1 score and AUC, RF (specificity = 0.99, sensitivity = 0.58, accuracy = 85.86%, F1 score = 0.9243, AUC = 0.549) outperformed the other methods. Both the classical (LR) and machine learning (ML) models’ performance has improved dramatically when important characteristics are extracted using the wrapper method. The LR method identified five significant features with a specificity, sensitivity, accuracy and F1 score of 0.87, 0.33, 87.12% and 0.9309. The region, whether the infant is a twin, and cesarean delivery were the three key features discovered by the DT and RF models, which were implemented using the wrapper technique. All three models had the identical F1 score of 0.9318. However, “child is twin” was recognized as a significant feature by the SVM, NB, and AB models, with an F1 score of 0.9315. Ultimately, with an F1 score of 0.9315, the XGBoost model recognized “child is twin” and “age at first sex” as relevant features. Random Forest again beat the other approaches in this instance.ConclusionsThe study reveals Wrapper method as the optimal feature selection technique. The ML method outperforms traditional methods, with Random Forest (RF) being the most effective predictive model for Low-Birth-Weight prediction. The study suggests that policymakers in Bangladesh can mitigate low birth weight newborns by considering identified risk factors.

The objective of this research is to apply machine learning techniques to optimize electric vehicle battery management and balance to attain maximum battery performance. Here, we will assess and compare the efficiency and accuracy of decision tree classifier, ANN, and Naive Bayes classifiers in: predicting two optimal charging and discharging battery management strategies, prediction of the battery’s performance and detecting any abnormalities in it. . In this context, one machine learning model will be proposed to have the most advantageous performance. The experimental results details that such findings are attained, and the precision rates, recall rates, and F1-scores attained by net models exceed 98%. Additionally, the decision tree, as well as the Naive Bayes classifier, have impressive performance, and their accuracy rates exceed 90%. Decision trees along with Naive Bayes classifiers have also important impacts on the identification of classification of battery’s responses through probabilistic classification in the former. Our findings have important implications for ensuring electric vehicle batteries are managed more properly for optimal performance. Additionally, based on the results obtained, they can be utilized to help relevant stakeholders in the EV industry and other industries with battery usage implement practical measures that optimize battery performance, increase battery life, and reduce the incurred operational expenses of electric vehicle fleets.

Naive Bayes Tree uses decision tree as the general structure and deploys naive Bayesian classifiers at leaves. The intuition is that naive Bayesian classifiers work better than decision trees when the sample data set is small. Therefore, after several attribute splits when constructing a decision tree, it is better to use naive Bayesian classifiers at the leaves than to continue splitting the attributes. In this paper, we propose a learning algorithm to improve the conditional probability estimation in the diagram of Naive Bayes Tree. The motivation for this work is that, for cost-sensitive learning where costs are associated with conditional probabilities, the score function is optimized when the estimates of conditional probabilities are accurate. The additional benefit is that both the classification accuracy and Area Under the Curve (AUC) could be improved. On a large suite of benchmark sample sets, our experiments show that the CLL tree outperforms the state-of-art learning algorithms, such as Naive Bayes Tree and naive Bayes significantly in yielding accurate conditional probability estimation and improving classification accuracy and AUC.

The coexistence problem occurs when a wireless body area network (WBAN) is close to other WBANs. The degraded communication performance may even threaten people’s life for medical applications. Therefore, a reliable method for coexistence state prediction is required to ensure this problem could be detected and handled in time. This chapter presents a machine learning based method for coexistence state prediction, in which decision tree (DT) and naive Bayes classifier (NBC) are considered as supervised learning methods. Firstly, the average packet error rate (PER) and other 11 features extracted from signal to interference plus noise ratio (SINR), which have less computational cost, are selected and improved to reflect human relative movement and interference strength for feature extraction. Then, a set of models based on DT and NBC are generated with various configurations of different features, the number of continuous SINRs and the decision function. Finally, all models are compared and the best one is selected for state prediction. In the experiment, the data for training and testing is collected by CC2530 2.4 GHz low-power transceivers. Simulation results show that, compared with NBC based method, the proposed method is better in terms of accuracy and timeliness.

Machine learning (ML) models are cost-effective tools that can establish a prognosis by considering clinical, genomic, and histological data from breast cancer patients. In this study, we evaluate the performance of 12 ML models in predicting the overall survival of breast cancer patients from “The Cancer Genome Atlas-Breast Cancer” database at 60 months of follow-up. The models evaluated were Logistic Regression (LR), Ridge Classifier (RC), Least Absolute Shrinkage and Selection Operator (LASSO), K-Nearest Neighbors (KNN), Naive Bayes (NB), Linear Discriminant Analysis (LDA), Decision Tree (DT), Multilayer Perceptron (MP), Stochastic Gradient Descent (SGD), Support Vector Machine (SVM), Random Forest (RF), and XGBoost (XGB). The most accurate models were NB (86.76%) and SGD (85.29%). The models with the highest sensitivity were SGD (93.75%), MP (91.67%), and NB (89.89%), while the models with the highest specificity were KNN (95.0%), DT (90.0%), and LASSO (85.0%). The models with the highest area under the ROC curve were LDA (90.83%), LASSO (90.1%), and LR (89.79%). The most relevant variables were “Previous Diagnosis of Cancer,” “Presence of Tumor,” “Ancillary Therapy,” and “Histology.” Our findings demonstrate that AI tools for predicting patient outcomes are a viable option when expensive prognostic genomic tools are not available. According to our results, NB and LASSO show promising performance in predicting the prognosis of breast cancer patients. Further studies with data from developing countries are needed to improve the performance of these tools.

We have analyzed 17 immunological parameters in 63 schizophrenia patients and 36 healthy volunteers. The parameters of humoral immunity, systemic level of the key cytokines of adaptive immunity, anti-inflammatory and pro-inflammatory cytokines, and other inflammatory markers were determined by enzyme immunoassay. Applied methods of machine learning covered the main group of approaches to supervised learning such as linear models (logistic regression), quadratic discriminant analysis (QDA), support vector machine (linear SVM, RBF SVM), k-nearest neighbors algorithm, Gaussian processes, naive Bayes classifier, decision trees, and ensemble models (AdaBoost, random forest, XGBoost). The importance of features for prediction from the best fold has been analyzed for the machine learning methods, which demonstrated the best quality. The most significant features were selected using 70% quantile threshold. The AdaBoost ensemble model with ROC AUC of 0.71±0.15 and average accuracy (ACC) of 0.78±0.11 has demonstrated the best quality on a 10-fold cross validation test sample. Within the frameworks of the present investigation, the AdaBoost model has shown a good quality of classification between the patients with schizophrenia and healthy volunteers (ROC AUC over 0.70) at a high stability of the results (σ less than 0.2). The most important immunological parameters have been established for differentiation between the patients and healthy volunteers: the level of some systemic inflammatory markers, activation of humoral immunity, pro-inflammatory cytokines, immunoregulatory cytokines and proteins, Th1 and Th2 immunity cytokines. It was for the first time that the possibility of differentiating schizophrenia patients from healthy volunteers was shown with the accuracy of more than 70% with the help of machine learning using only immune parameters.The results of this investigation confirm a high importance of the immune system in the pathogenesis of schizophrenia.

The design complexity and outsourcing trend of modern integrated circuits (ICs) have increased the chance for adversaries to implant hardware Trojans (HTs) in the development process. To effectively defend against this hardware-based security threat, many solutions have been reported in the literature, including dynamic and static techniques. However, there is still a lack of methods that can simultaneously detect and diagnose HT circuits with high accuracy and low time complexity. Therefore, to overcome these limitations, this paper presents an HT detection and diagnosis method for gate-level netlists (GLNs) based on different machine learning (ML) algorithms. Given a GLN, the proposed method first partitions it into several circuit cones and extracts seven HT-related features from each cone. Then, we repeat this process for the sample GLN to construct a dataset for the next step. After that, we use K-Nearest Neighbor (KNN), Decision Tree (DT) and Naive Bayes (NB) to classify all circuit cones of the target GLN. Finally, we determine whether each circuit cone is HT-implanted through the label, completing the HT detection and diagnosis for target GLN. We have applied our method to 11 GLNs from ISCAS’85 and ISCAS’89 benchmark suites. As shown in experimental results of the three ML algorithms used in our method: (1) NB costs shortest time and achieves the highest average true positive rate (ATPR) of 100%; (2) DT costs longest time but achieve the highest average true negative rate (ATNR) of 98.61%; (3) Compared to NB and DT, KNN costs a slightly longer time than NB but the ATPR and ATNR values are approximately close to DT. Moreover, it can also report the possible implantation location of a Trojan instance according to the detecting results.

In practical classification problems, imbalanced class distribution is very common. Traditional classification methods usually aim at optimizing the overall accuracy, which leads to classification decisions biased towards categories with a large number of samples. Similar to most traditional classifiers, Naive Bayes (NB) is sensitive to the class prior distribution and has poor recognition performance for minority class samples, which restricts its application in imbalanced classification. Therefore, we propose a Bayesian classifier based on distance-corrected multivariate kernel density estimation (BC-DMKDE). BC-DMKDE relaxes the assumptions regarding conditional independence and the normality of continuous variables in NB classifiers, by incorporating the multivariate kernel density estimation (MKDE). We also incorporate a distance-correction mechanism utilizing kernel density estimation, specifically to alleviate the impact of class imbalance and overlapping data samples. Across 18 real-world benchmarks, BC-DMKDE exhibits superior classification performance and robustness, outperforming existing methods at both the algorithm and data levels.