Secure Efficient Federated KNN for Recommendation Systems

Deep supervised hashing hash has been widely utilized in large-scale image retrieval due to its lightweight storage and fast search speed. The distribution of features from the existing hashing methods has been inevitably distorted from the original feature distribution, which resulted in the performance decline on image retrieval. With the constraints of the Euclidian metric, the distortion can be mitigated to certain extent. However, the Euclidian distance is sensitive to the norm of the features. On the point, we propose a novel deep hashing method, called cosine metric supervised deep hashing (CMDH), to perform hash learning by incorporating cosine metric and category loss. CMDH uses the cosine metric to effectively mitigate the impact caused by the diverse sample vector norms on retrieval performance. In addition, joint deep hash learning constructs binary coding over cosine loss and category loss, which further alleviates distortion in binary feature learning for efficient retrieval.KeywordsCosine metricClassification informationSupervised deep hashingImage retrieval

The nearest neighbor search is an essential operation for many computer vision, data mining, and machine learning problems. Since it is so widely used, the nearest neighbor search should be as fast as possible. This paper explores lower bound-based approaches to speed up the exact nearest neighbor search in high dimensional Euclidean space. We compute the lower bound of Euclidean Distance by using the block vectors and Cauchy–Schwartz inequality. The proposed lower bound is calculated efficiently and is close to the real Euclidean Distance. Besides, the preprocessing step of the proposal has linear time complexity. Given a query, during the procedure of identifying the nearest neighbor, our method can eliminate many expensive actual distance computations using the lower bound to approximate Euclidean Distance. In addition, we develop a multilevel lower bound strategy, which calculates the lower bound step by step and utilizes the multistep filtering mechanism to improve the searching process further. Theoretical analysis is provided to show that the proposals can guarantee to obtain the same result as the brute-force search. Comprehensive experiments on 16 public data sets collected from various domains demonstrate that our approach performs well in finding the exact nearest neighbor compared to related competitors. The experimental results also illustrate that the multilevel lower bound strategy is effective.

Nearest neighbor (NN) searches represent an important class of queries in geographic information systems (GIS). Most nearest neighbor algorithms rely on static distance information to compute NN queries (e.g., Euclidean distance or spatial network distance). However, the final goal of a user when performing an NN search is often to travel to one of the points of the search result. In this case, finding the nearest neighbors in terms of travel time is more important than the actual distance. In the existing NN algorithms dynamic real-time events (e.g., traffic congestions, detours, etc.) are usually not considered and hence the pre-computed nearest neighbor objects may not accurately reflect the shortest travel time. In this paper we propose a novel travel time network that integrates both spatial networks and real-time traffic event information. Based on this foundation of the travel time network, we develop a local-based greedy nearest neighbor algorithm and a global-based adaptive nearest neighbor algorithm that both utilize real-time traffic information to provide adaptive nearest neighbor search results. We have performed a theoretical analysis and simulations to verify our methods. The results indicate that our algorithms remarkably reduce the travel time compared with previous nearest neighbor solutions.

The classification of steel surface defects plays a very important role in analyzing their causes to improve manufacturing process and eliminate defects. However, defective samples are very scarce in actual production, so using very few samples to construct a good classifier is a challenge to be addressed. If the layer number of the model with proper depth is increased, the model accuracy will decrease (not caused by overfit), and the training error as well as the test error will be very high. This is called the degradation problem. In this paper, we propose to use feature extraction + feature transformation + nearest neighbors to classify steel surface defects. In order to solve the degradation problem caused by network deepening, the three feature extraction networks of Residual Net, Mobile Net and Dense Net are designed and analyzed. Experiment results show that in the case of a small sample number, Dense block can better solve the degradation problem caused by network deepening than Residual block. Moreover, if Dense Net is used as the feature extraction network, and the nearest neighbor classification algorithm based on Euclidean metric is used in the new feature space, the defect classification accuracy can reach 92.33% when only five labeled images of each category are used as the training set. This paper is of some guiding significance for surface defect classification when the sample number is small.

Our purpose was to evaluate the potential usefulness of the nearest neighbor case which was assumed to be the similar case in a CAD scheme for determining the histological classification of clustered microcalcifications. Our database consisted of current and previous magnification mammograms obtained from 93 patients before and after three-month follow-up examination. It included 11 invasive carcinomas, 19 noninvasive carcinomas of the comedo type, 25 non-invasive carcinomas of the noncomedo type, 23 mastopathies, and 15 fibroadenomas. Six objective features on clustered microcalcifications were first extracted from each of the current and the previous images. The nearest neighbor case was then identified by the Euclidean distance in the previous and current feature-space. The histological classification of an unknown new case in question was assumed to be the same as that of the nearest neighbor case which has the shortest Euclidean distance in our database. The classification accuracies were 90.9% for invasive carcinoma, 89.5% for noninvasive carcinoma of the comedo type, 96.0% for noninvasive carcinoma of the noncomedo type, 82.6% for mastopathy, and 93.3% for fibroadenoma. These results were substantially higher than those with our previous CAD scheme. The nearest neighbor criterion was useful in a CAD scheme for determining the histological classification.

We consider popular dimension reduction techniques that project data on a low dimensional subspace. They include Principal Component Analysis, Column Subset Selection, and Johnson-Lindenstrauss projections. These techniques have been classically used to efficiently compute various approximations. We propose the following three-step procedure for enhancing the accuracy of such approximations: 1. Unknown quantities in the approximation are replaced with random variables. 2. The Maximum Entropy method is applied to infer the most likely probability distribution. 3. Expected values of the random variables are used to compute the enhanced estimates. Our use of the Maximum Entropy method requires knowledge of vector norms that can be easily computed during the dimension reduction. We demonstrate significant enhancements in average accuracy for Euclidean distance and Mahalanobis distance, and improvements in evaluating k-nearest neighbors and k-furthest neighbors by using the enhanced Euclidean distance formula.

Instance-based learning techniques use a set of stored training instances to classify new examples. The most common such learning technique is the nearest neighbor method, in which new instances are classified according to the closest training instance. A critical element of any such method is the metric used to determine distance between instances. Euclidean distance is by far the most commonly used metric; no one, however, has systematically considered whether a different metric, such as Manhattan distance, might perform equally well on naturally occurring data sets. Some evidence from psychological research indicates that Manhattan distance might be preferable in some circumstances. This paper examines three different distance metrics and presents experimental comparisons using data from three domains: malignant cancer classification, heart disease diagnosis, and diabetes prediction. The results of these studies indicate that the Manhattan distance metric works works quite well, although not better than the Euclidean metric that has become a standard for machine learning experiments. Because the nearest neighbor technique provides a good benchmark for comparisons with other learning algorithms, the results below include a number of such comparisons, which show that nearest neighbor, using any distance metric, compares quite well to other machine learning techniques.

Soil moisture (SM) is a critical parameter in maintaining the balance of water cycle and energy budgets between climate system and the Earth's environment. Generalized regression neural network (GRNN) has been substantially verified as a powerful model for SM estimation due to the ability of capturing complex, non-linear relationships between predictors and responses. However, GRNN builds a full adjacency matrix using Gaussian kernel, which is computationally expensive and may ignore the local structure. In addition, it is laborious to optimize the “spread” parameter. To overcome the above issues, we propose an enhanced generalized regression neural network (EGRNN) for SM estimation, where two main adaptations are made. On the one hand, the City block distance instead of the Euclidean distance is used for building Gaussian kernel. On the other hand, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> -nearest neighbors ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> -NN) is adopted to yield an empirically sparse adjacency matrix. As the key advantage, the proposed EGRNN weakens the sensitivity to outliers since large differences are weighted more heavily by using Euclidean distance than City block distance. Another advantage is that EGRNN models more local and discriminant information in the pattern layer since only the data points within neighbors are connected by using <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> -NN. Experiments conducted in the Qinghai-Tibet Plateau (QTP) demonstrate that; 1) EGRNN outperforms the other four neural network models, with R = 0.9485 and RMSE = 0.0325 cm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> /cm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> ; 2) It can well capture spatial-temporal dynamics and has higher consistent with the in-situ measurements; 3) It adapts well to different in-situ networks and has better generalization performance.

This paper presents the prediction of the output power of a solar power plant based on the nearest neighbor regression method. Data from a 50 MW solar power plant in China with solar radiation, ambient temperature, pressure, and humidity values and corresponding output power values from 01/01/2019 to 12/31/2020 with 15-minute intervals were used to create the output power forecast. Before using the data, they were cleaned of outliers using the standard interquartile range method, data points were divided into test and training groups, and feature scaling was applied using the standardization method to correctly calculate the Euclidean distance between data points. The application of clustering of weather parameters by the k-means method is proposed, which allows for individual selection of the number of neighbors for each cluster and to exclude the influence of points of one cluster on the number of neighbors of another cluster. The number of clusters is selected by determining the silhouette coefficient, the training group of weather parameters with their corresponding output power values is distributed among clusters based on the Euclidean distance to the centroid of the clusters. The test group of weather parameters is divided into clusters, after which forecasting by the method of nearest neighbors takes place within each cluster separately. The output power of the solar power plant is calculated as the weighted arithmetic average of the neighbors of each point of the test group. At the end of the algorithm, the sequence of points of weather parameters of the test group is restored and a time series of the output power forecast is created. The proposed algorithm made it possible to reduce the MSE, RMSE, MAPE, MAE forecast errors for 1 day by 0.5348, 0.2265, 0.38%, 0.1448, respectively, for 7 days, the errors became smaller by 0.1992, 0.0384, 0.1%, 0.0193, respectively. As a result, the relative error in forecasting for 24 hours is 4.22%.

The fractional Fourier transform (FRFT) has the mixed time and frequency characteristics of signals. And FRFT is a powerful and effective tool for time-varying and nonstationary signal processing. To address face recognition problem, an approach based on 2D discrete fractional Fourier transform (2D-DFRFT) is proposed in this paper. First of all, the 2D-DFRFT of a facial image is computed. Then Principal Component Analysis (PCA) and Fisher Linear Discriminant Analysis (FLDA) are combined to perform discriminative feature extraction. Then the nearest neighbor (NN) classifier is applied to perform face recognition, based on Euclidean distance. Experimental results on ORL face database indicate that the proposed approach obtains good recognition effects.

Nearest neighbor (NN) techniques are commonly used in remote sensing, pattern recognition, and statistics to classify objects into a predefined number of categories based on a given set of predictors. These techniques are particularly useful in those cases exhibiting a highly nonlinear relationship between variables. In most studies, the distance measure is adopted a priori. In contrast, we propose a general procedure to find Euclidean metrics in a low-dimensional space (i.e., one in which the number of dimensions is less than the number of predictor variables) whose main characteristic is to minimize the variance of a given class label of all those pairs of points whose distance is less than a predefined value. k-NN is used in each embedded space to determine the possibility that a query belongs to a given class label. The class estimation is carried out by an ensemble of predictions. To illustrate the application of this technique, a typical land cover classification using a Landsat-5 Thematic Mapper scene is presented. Experimental results indicate substantial improvement with regard to the classification accuracy as compared with approaches such as maximum likelihood, linear discriminant analysis, standard k-NN, and adaptive quasi-conformal kernel k-NN.

As an improved method of k-nearest neighbor classification, the local mean-based nonparametric classifier had the ability to resist the effects of noise and classify unbalanced data. When selecting the nearest k samples and calculating the distance between the test samples and the local mean-based vectors, it always used Euclidean distance. However, for multi-dimensional data, using Euclidean distance which focused on the difference of the value to determine whether two vectors was similar was not so accurate. To solve this problem, a new local mean-based nonparametric classification method was proposed in this paper. It used the cosine distance which focused more on the difference of the dimension to select the k nearest neighbors and compute the distance between the test samples and the local mean-based vect ors. The new local mean-based nonparametric classification method was tested on the UCI datasets: Iris and Wine for different values of k in different test data set, the simulation results show that it outperforms the existing local mean-based nonparametric classifier.

BackgroundA new learning-based patient similarity measurement was proposed to measure patients’ similarity for heterogeneous electronic medical records (EMRs) data.MethodsWe first calculated feature-level similarities according to the features’ attributes. A domain expert provided patient similarity scores of 30 randomly selected patients. These similarity scores and feature-level similarities for 30 patients comprised the labeled sample set, which was used for the semi-supervised learning algorithm to learn the patient-level similarities for all patients. Then we used the k-nearest neighbor (kNN) classifier to predict four liver conditions. The predictive performances were compared in four different situations. We also compared the performances between personalized kNN models and other machine learning models. We assessed the predictive performances by the area under the receiver operating characteristic curve (AUC), F1-score, and cross-entropy (CE) loss.ResultsAs the size of the random training samples increased, the kNN models using the learned patient similarity to select near neighbors consistently outperformed those using the Euclidean distance to select near neighbors (all P values < 0.001). The kNN models using the learned patient similarity to identify the top k nearest neighbors from the random training samples also had a higher best-performance (AUC: 0.95 vs. 0.89, F1-score: 0.84 vs. 0.67, and CE loss: 1.22 vs. 1.82) than those using the Euclidean distance. As the size of the similar training samples increased, which composed the most similar samples determined by the learned patient similarity, the performance of kNN models using the simple Euclidean distance to select the near neighbors degraded gradually. When exchanging the role of the Euclidean distance, and the learned patient similarity in selecting the near neighbors and similar training samples, the performance of the kNN models gradually increased. These two kinds of kNN models had the same best-performance of AUC 0.95, F1-score 0.84, and CE loss 1.22. Among the four reference models, the highest AUC and F1-score were 0.94 and 0.80, separately, which were both lower than those for the simple and similarity-based kNN models.ConclusionsThis learning-based method opened an opportunity for similarity measurement based on heterogeneous EMR data and supported the secondary use of EMR data.

To solve the problems of lean, occlusion, and unbalance exposure, a new strategy of facial expression recognition based on template matching is proposed in the paper. The strategy firstly obtains the characteristics of half-facial expressions by using Gabor wavelet, and then calculates the picture's Euclidean distance with the standard library; at last, classify expressions by using the method of K nearest neighbor. Experiments show that the strategy can improve the rate of facial expression recognition under the condition of without increasing in the expression library. The experiments prove that the Euclidean distance becomes small; and the amount of computing data reduces more.

The continuous K nearest neighbor (CKNN) query is an important type of query that finds continuously the KNN to a query point on a given path. We focus on moving queries issued on stationary objects in Spatial Network Database (SNDB) The result of this type of query is a set of intervals (defined by split points) and their corresponding KNNs. This means that the KNN of an object traveling on one interval of the path remains the same all through that interval, until it reaches a split point where its KNNs change. Existing methods for CKNN are based on Euclidean distances. In this paper we propose a new algorithm for answering CKNN in SNDB where the important measure for the shortest path is network distances rather than Euclidean distances. We propose DAR and eDAR algorithms to address CKNN queries based on the progressive incremental network expansion (PINE) technique. Our experiments show that the eDAR approach has better response time, and requires fewer shortest distance computations and KNN queries than approaches that are based on VN3 using IE.