KNN Algorithm with Data-Driven k Value

Algorithms based on Nested Generalized Exemplar (NGE) theory (Salzberg, 1991) classify new data points by computing their distance to the nearest “generalized exemplar” (i.e., either a point or an axis-parallel rectangle). They combine the distance-based character of nearest neighbor (NN) classifiers with the axis-parallel rectangle representation employed in many rule-learning systems. An implementation of NGE was compared to the k-nearest neighbor (kNN) algorithm in 11 domains and found to be significantly inferior to kNN in 9 of them. Several modifications of NGE were studied to understand the cause of its poor performance. These show that its performance can be substantially improved by preventing NGE from creating overlapping rectangles, while still allowing complete nesting of rectangles. Performance can be further improved by modifying the distance metric to allow weights on each of the features (Salzberg, 1991). Best results were obtained in this study when the weights were computed using mutual information between the features and the output class. The best version of NGE developed is a batch algorithm (BNGE FWMI) that has no user-tunable parameters. BNGE FWMI's performance is comparable to the first-nearest neighbor algorithm (also incorporating feature weights). However, the k-nearest neighbor algorithm is still significantly superior to BNGE FWMI in 7 of the 11 domains, and inferior to it in only 2. We conclude that, even with our improvements, the NGE approach is very sensitive to the shape of the decision boundaries in classification problems. In domains where the decision boundaries are axis-parallel, the NGE approach can produce excellent generalization with interpretable hypotheses. In all domains tested, NGE algorithms require much less memory to store generalized exemplars than is required by NN algorithms.

k-nearest neighbor and centroid-based classification algorithms are frequently used in text classification due to their simplicity and performance. While k-nearest neighbor algorithm usually performs well in terms of accuracy, it is slow in recognition phase. Because the distances/similarities between the new data point to be recognized and all the training data need to be computed. On the other hand, centroid-based classification algorithms are very fast, because only as many distance/similarity computations as the number of centroids (i.e. classes) needs to be done. In this paper, we evaluate the performance of centroid-based classification algorithm and compare it to nearest mean and nearest neighbor algorithms on 9 data sets. We propose and evaluate an improvement on centroid based classification algorithm. Proposed algorithm starts from the centroids of each class and increases the weight of misclassified training data points on the centroid computation until the validation error starts increasing. The weight increase is done based on the training confusion matrix entries for misclassified points. Vie proposed algorithm results in smaller test error than centroid-based classification algorithm in 7 out of 9 data sets. It is also better than 10-nearest neighbor algorithm in 8 out of 9 data sets. We also evaluate different similarity metrics together with centroid and nearest neighbor algorithms. We find out that, when Euclidean distance is turned into a similarity measure using division as opposed to exponentiation. Euclidean-based similarity can perform almost as good as cosine similarity.

The security and integrity of computer systems and networks highly depend on malware detection. In the realm of malware detection, the K-Nearest Neighbors (KNN) algorithm is a well-liked and successful machine learning algorithm. However, the choice of an acceptable distance metric parameter has a significant impact on the KNN algorithm's performance. This study tries to improve malware detection by adjusting the KNN algorithm's distance metric parameter. The distance metric greatly influences the similarity or dissimilarity between instances in the feature space. The KNN algorithm for malware detection can be more accurate and effective by carefully choosing or modifying the distance metric. This paper analyzes multiple distance metrics, including Minkowski distance, Manhattan distance, and Euclidean distance. These metrics account for the traits of malware samples while capturing various aspects of similarity. The effectiveness of the KNN algorithm is evaluated using the MalMem-2022 malware dataset, and the results are broken down into these three-distance metrics. The experimental findings show that, among the three distance metric parameters, the Euclidean and Minkowski distance metric parameters considerably produced the best outcomes with binary classification. While with multiclass classification, the KNN algorithm has achieved the highest outcomes using Manhattan distance.

Abstract. This system was developed to determine the accuracy of sentiment analysis on Twitter regarding the COVID-19 issue using the Nazief-Adriani and Porter stemmers with TF-IDF weighting, along with a classification process using K-Nearest Neighbor (KNN) that resulted in a comparison of 48.24% for Nazief-Adriani and 48.24% for Porter. Purpose: This research aims to determine the accuracy of the Nazief-Adriani and Porter stemmer algorithms in performing text preprocessing using a dataset from Indonesian-language Twitter. This research involves word weighting using TF-IDF and classification using the K-Nearest Neighbor (KNN) algorithm. Methods/Study design/approach: The experimentation was conducted by applying the Nazief-Adriani and Porter stemmer algorithm methods, utilizing data sourced from Twitter related to COVID-19. Subsequently, the data underwent text preprocessing, stemming, TF-IDF weighting, accuracy testing of training and testing data using K-Nearest Neighbor (KNN) algorithm, and the accuracy of both stemmers was calculated employing a confusion matrix table. Result/Findings: This study obtained reasonably accurate results in testing the Nazief-Adriani stemmer with an accuracy of 50.98%, applied to sentiment analysis of COVID-19-related Twitter data using the Indonesian language. As for the accuracy of the Porter stemmer, it achieved an accuracy rate of 48.24%. Novelty/Originality/Value: Feature selection is crucial in stemmer accuracy testing. Therefore, in this study, feature selection is carried out using the Nazief-Adriani and Porter stemmers for testing purposes, and the accuracy data classification is conducted using the K-Nearest Neighbor (KNN) algorithm

For the problem that the prediction accuracy of real-valued attribute data is not high, a modeling method named PR-KNN (Polynomial Regression and K Nearest Neighbor) is proposed, which is based on combination of KNN (K Nearest Neighbor) algorithm and Polynomial Regression model. Firstly, K nearest decision attribute values in training samples are selected by using KNN algorithm. Secondly, these K nearest decision attribute values are modeled by using Polynomial Regression method. And this method is applied to aftershock prediction. Experimental data are the sequence data of aftershocks with magnitude greater than or equal to 4.0 from Wenchuan earthquake. Comparing with traditional KNN regression algorithm and Distance-Weighted KNN regression algorithm, experimental results show that the maximum relative error predicted by PR-KNN reduces by 6.012% and 7.751% respectively, and maximum absolute error reduces by 0.367 and 0.473 respectively.

The availability of a lot of existing Standard Operating Procedures (SOP) document information, users often need time to find SOPs that fit their preference. Therefore, this requires a recommendation system based on user content consumption by personalized usage logs to support the establishment of SOP documents managed according to user preferences. The k-nearest neighbor (KNN) algorithm is used to identify the most relevant SOP document for the user by utilizing implicit feedback based on extraction data by monitoring the document search behavior. From the research results obtained 5 classifications as parameters, with a final value of 3:2 ratio that shows the best distance value with the majority of labels according to the concept of calculation KNN algorithm that sees from the nearest neighbor in the dataset. This shows the precision of applying the KNN algorithm in determining SOP documents according to user preferences based on implicit feedback resulting in 80% presentation for SOPs corresponding to profiles and 20% for SOPs that do not fit the user profile. To establish SOP documents to show more accurate results, it should be used in a broad SOP management system and utilize implicit feedback with parameters not only in search logs and more on performance evaluation evaluations.

Sales data analysis is an important aspect of modern businesses, as it enables them to gain insights into customer preferences and behavior, optimize marketing strategies, and improve revenue. With the rise of big data and machine learning, businesses can now leverage these tools to analyze sales data and gain even deeper insights. This paperfocuses on analyzing sales data using machine learning algorithms, with a specific emphasis on the K-Nearest Neighbors (KNN) algorithm. KNN is a popular machine learning algorithm used in classification and regression problems, and it works by identifying the K closest data points to a new observation and predicting its class or value based on the majority class or average value of its nearest neighbors. The dataset used in this paperis from a retail store and includes information on product type, sales channel, customer demographics, and sales volume over a period of several years. The paper will use Python and various machine learning libraries to perform data preprocessing, exploratory data analysis, feature engineering, and modeling. Specifically, KNN will be used to develop a predictive model for sales forecasting, which can help businesses optimize their inventory and production planning. The paper will also use clustering techniques, such as K-Means, to identify customer segments based on their purchasing behavior and demographics, and association rule mining to identify product affinities and recommend complementary products. The results of this analysis can help businesses gain a deeper understanding of their sales data and make data-driven decisions to improve their bottom line. By leveraging the power of KNN and other machine learning algorithms, businesses can gain insights into customer behavior and preferences, optimize their marketing strategies, and improve their overall revenue. Keywords: KNN Algorithm, Data Analytics, Machine Learning, Support Vector Machine.

This research aims to analyze the application of the K-Nearest Neighbor (KNN) algorithm in classifying regional structures in Medan City. Medan, one of the largest cities in Indonesia, has a variety of characteristics of regional structure that requires an appropriate analysis approach for spatial management and spatial planning. The KNN algorithm was chosen because of its ability to categorize data based on its proximity to other data points, which is very suitable for the needs of spatial planning and management. other data points, which is very suitable for the needs of regional classification analysis. In this research, the data used includes various attributes of the regional structure such as structure attributes such as population density, land use, and infrastructure in each sub-district. infrastructure in each sub-district in Medan City. In this research, the method used is statistical data processing statistical data processing to group areas with similar characteristics, using the KNN algorithm as a classification method. The classification process process involves selecting the right parameters, calculating the distance between data points, and selecting the optimal number of nearest neighbors. data points, as well as selecting the optimal number of nearest neighbors. The expected results The expected results of this analysis will provide a clear picture of the distribution pattern of the distribution pattern of the regional structure in Medan City, as well as assisting in the planning and development of a more efficient and directed city. The accuracy of the KNN model in classifying the regions will also be compared with other algorithms to assess its effectiveness and reliability in the context of this study.

Ornamental cork fish is a type of fish that is in great demand among the public as an ornamental fish. Ornamental cork fish have various types and colors; each variation has its own name and is a selling point among ornamental cork fish lovers. With a good motif, ornamental cork fish will have an expensive market value. However, for the most part, there are still many who do not know for sure what type of ornamental cork fish is included in the variation type classification because the colors are varied and seem similar. Because of this, this research created a system that can classify types of ornamental cork fish automatically based on data while still paying attention to the level of accuracy of the classification. The algorithm used for the initial classification process is KNN, which is chosen for its accuracy comparison level value. This algorithm does not consider the weight of each data point to be classified. The data processing process carried out only looks at the highest number of classes, which becomes the benchmark for labels from the classification results. In the classification process method using the KNN algorithm, there are still shortcomings in the classification process, so this research carried out a process of comparing classification accuracy using the Weight-KNN algorithm to increase the classification accuracy value. The process of the Weight-KNN algorithm stages is to carry out classification based on nearest neighbors first but still paying attention to the weight of each data. So that the classification process of determining the type of ornamental cork fish variation will be more accurate. Based on the results of experiments conducted, this research will focus on comparing the classification results between the KNN and Weight-KNN algorithms on ornamental cork fish. The results obtained state that the Weight-KNN algorithm has a higher level of accuracy with a weight of 83.6%, whereas using the KNN algorithm, it is only 80.6%.

The support vector machine (SVM) combined with the K-nearest neighbor (KNN) algorithm is applied to the 3D image recognition algorithm to cope with the issues of 3D image recognition in the image processing process. In this paper, the KNN algorithm is used to extend the eigenvalues of the residual network from frequency to complex plane according to the eigenvalues of 3D images, and the relevant data are extracted from the images by using the corresponding separation rules of the SVM. Subsequently, a sparse observation model is established based on the SVM combined with the KNN, and the 3D image processing issue is converted to a residual network visualization issue of 3D image recognition by establishing a parameterized database. Meanwhile, detection of high-resolution distance of 3D images is conducted, and the variational inference method is used to visualize and analyze the 3D images accordingly. Finally, through the experimental study, it can be known that the method of SVM combined with the KNN algorithm can reach an accuracy of up to 94.18% in the dataset of the Princeton 3D polygonal models. After comparison with the existing SVM-KNN method and deep learning, it can be concluded that the method put forward in this paper has a higher recognition rate and stronger anti-interference capacity, and the algorithm has a superior convergence speed with less adjustment parameters.

Previous scholars always made an effort to make various formulations that were used to categorize and calcify hadith. At present, the process of categorization or classification is facilitated by the process of text mining technology. In the study of text mining itself, there are various kinds of tools and methods or algorithms that can be used and also help provide maximum results in the process of mining information from a text. An example is the Decision Tree C4.5 and K-Nearest Neighbor algorithm. Based on that, the author wants to make research and this final project to compare the performance resulting from the classification process of text documents using Decision Tree C4.5 and K-Nearest Neighbor algorithm for the classification of Imam At- Tirmidzi hadith. With this research, it is expected to be knowledgeable about the process of classifying text documents along with the performance of the two algorithms. Based on testing that has been done, the Decision Tree C4.5 algorithm produces an average accuracy value of 70.53% with an average processing time of 0.083 seconds. While the K-Nearest Neighbor algorithm produces an average accuracy value of 66.36% with an average processing time of 0,03 seconds.

Blood pressure (BP) is an important parameter for the early detection of heart disease because it is associated with symptoms of hypertension or hypotension. A single photoplethysmography (PPG) method for the classification of BP can automatically analyze BP symptoms. Users can immediately know the condition of their BP to ensure early detection. In recent years, deep learning methods have presented outstanding performance in classification applications. However, there are two main problems in deep learning classification methods: classification accuracy and time consumption during training. We attempt to address these limitations and propose a method for the classification of BP using the K-nearest neighbors (KNN) algorithm based on PPG. We collected data for 121 subjects from the PPG–BP figshare database. We divided the subjects into three classification levels, namely normotension, prehypertension, and hypertension, according to the BP levels of the Joint National Committee report. The F1 scores of these three classification trials were 100%, 100%, and 90.80%, respectively. Hence, it is validated that the proposed method can achieve improved classification accuracy without additional manual pre-processing of PPG. Our proposed method achieves higher accuracy than convolutional neural networks (deep learning), bagged tree, logistic regression, and AdaBoost tree.

Machine learning methods are widely used in automated technologies. Classification prediction is a machine learning based on data mining. Today, many technological devices can make new predictions by gaining experience from past data with machine learning methods. Machine learning is widely studied in two types, supervised and unsupervised. The limits of the objectives in supervised learning are predetermined. In unsupervised learning, there are no predetermined targets. In this learning, the machines are required to determine the targets automatically. Prediction process is one of the basic components of machine learning. Machines need to use some algorithms in order to perform the prediction process on the basis of data mining. k nearest neighbor (kNN), Naive Bayes (NB), Decision Tree (DT) and Support Vector Machine (SVM) algorithms are used mostly. k nearest neighbor (kNN), Naive Bayes (NB), Decision Tree (DT) and Support Vector Machine (SVM) algorithms are used mostly. These algorithms can be applied with the help of some tools on data sets. In this study, kNN algorithm was used to estimate Iris data set classification using Orange tool. The success of the KNN algorithm depends on using the correct attribute and changing the optimum k value. As a result of the tests, it was determined that when the k neighbor value was selected as 15, it was the most suitable k neighbor value, providing 98.67% classification prediction success in the Iris dataset.

The presence of missing data is a common problem that affects almost all clinical datasets. Since most available data mining and machine learning algorithms require complete datasets, accurately imputing (i.e. "filling in") the missing data is an essential step. This paper presents a methodology for the missing data imputation of longitudinal clinical data based on the integration of linear interpolation and a weighted K-Nearest Neighbours (KNN) algorithm. The Maximal Information Coefficient (MIC) values among features are employed as weights for the distance computation in the KNN algorithm in order to integrate intra- and inter-patient information. An interpolation-based imputation approach was also employed and tested both independently and in combination with the KNN algorithm. The final imputation is carried out by applying the best performing method for each feature. The methodology was validated on a dataset of clinical laboratory test results of 13 commonly measured analytes of patients in an intensive care unit (ICU) setting. The performance results are compared with those of 3D-MICE, a state-of-the-art imputation method for cross-sectional and longitudinal patient data. This work was presented in the context of the 2019 ICHI Data Analytics Challenge on Missing data Imputation (DACMI).

Aim: Prediction of coronary illness utilizing novel Novel k nearest neighbor (KNN) and contrasting its accuracy with decision tree algorithm. Materials and Methods: Two gatherings are proposed for foreseeing the accuracy (%) of coronary illness. To be specific, novel Novel k nearest neighbor and decision tree algorithm. Here we take 20 examples each for assessment and look at. The sample size was calculated using G power with pretest power at 80% and the alpha of 0.05 value. Result: The decision tree gives better accuracy (84.95%) contrasted with the Novel k nearest neighbor accuracy (76.29 %). Along these lines the factual meaning of the decision tree is superior to the novel k nearest neighbor algorithm with significance value of 0.115. Conclusion: From the outcome, it may very well be inferred that the decision tree helps in anticipating the coronary illness with more precision contrasted with the novel k nearest neighbor algorithm.