Private Distributed K-Means Clustering on Interval Data

Distributed K-Means clustering guaranteeing local differential privacy

In this paper, we study Local Information Privacy (LIP). As a context-aware privacy notion, LIP relaxes the de facto standard privacy notion of local differential privacy (LDP) by incorporating prior knowledge and therefore achieving better utility. We study the relationships between LIP and some of the representative privacy notions including LDP, mutual information and maximal leakage. We show that LIP provides strong instance-wise privacy protection compared to other context-aware privacy notions. Moreover, we present some useful properties of LIP, including post-processing, linkage, composability, transferability and robustness to imperfect prior knowledge. Then we study a general utility-privacy tradeoff framework, under which we derive LIP based privacy-preserving mechanisms for both discrete and continuous-valued data. Three types of perturbation mechanisms are studied in this paper: 1) randomized response (RR), 2) random sampling (RS) and 3) additive noise (AN) (e.g., Gaussian mechanism). Our privacy mechanisms incorporate the prior knowledge into the perturbation parameters so as to enhance utility. Finally, we present a comprehensive set of experiments on real datasets to illustrate the advantage of context-awareness and compare the utility-privacy tradeoffs provided by different mechanisms.

Federated learning (FL) allows to train a massive amount of data privately due to its decentralized structure. Stochastic gradient descent (SGD) is commonly used for FL due to its good empirical performance, but sensitive user information can still be inferred from weight updates shared during FL iterations. We consider Gaussian mechanisms to preserve local differential privacy (LDP) of user data in the FL model with SGD. The trade-offs between user privacy, global utility, and transmission rate are proved by defining appropriate metrics for FL with LDP. Compared to existing results, the query sensitivity used in LDP is defined as a variable, and a tighter privacy accounting method is applied. The proposed utility bound allows heterogeneous parameters over all users. Our bounds characterize how much utility decreases and transmission rate increases if a stronger privacy regime is targeted. Furthermore, given a target privacy level, our results guarantee a significantly larger utility and a smaller transmission rate as compared to existing privacy accounting methods.

Local differential privacy federated learning has attracted wide attention because it can solve the problem of data islands without damaging data privacy, but it faces data heterogeneity problems on the client side. Existing solutions commonly cluster clients with similar data distributions into the same training groups by leveraging clients’ model parameters, thus mitigating the impact of data heterogeneity on federated learning performance. However, model parameters, as an indirect representation for client data, often fail to accurately reflect the true data distribution, resulting in inaccurate client groupings. Furthermore, these solutions perturb high-dimensional parameter vectors dimension-by-dimension to protect client data privacy, which introduces substantial LDP noise that significantly further compromises the client grouping accuracy. To tackle these challenges, we propose PCFed-LDP, a privacy-preserving clustered federated learning framework that improves the federated learning performance while protecting client data and satisfying LDP in heterogeneous environments. Specifically, we introduce a client clustering method based on geometric properties of client data subspaces, which conducts label grouping-based principal angle analysis on client data subspaces to accurately capture the similarities in client data distributions, thereby enabling precise client grouping. To reduce the amount of introduced LDP noise, we design an adaptive noise addition method that utilizes the Haar wavelet technique to decouple the relationship between the noise amount and vector dimensionality, and employs noise error minimization strategy-based vector segmentation to inject LDP noise with finer granularity. Theoretical analysis and experiments on real datasets demonstrate that our solution not only satisfies the constraints of local differential privacy but also outperforms state-of-the-art methods.

Local Differential Privacy (LDP) has been widely recognized as a powerful tool for providing a strong theoretical guarantee of data privacy to data contributors against an untrusted data collector. Under a typical LDP scheme, each data contributor independently randomly perturbs their data before submitting them to the data collector, which in turn infers valuable statistics about the original data from received perturbed data. Common to existing LDP mechanisms is an inherent trade-off between the level of privacy protection and data utility in the sense that strong data privacy often comes at the cost of reduced data utility. Frequency estimation based on Randomized Response (RR) is a fundamental building block of many LDP mechanisms. In this paper, we propose a novel Joint Randomized Response (JRR) mechanism based on correlated data perturbations to achieve locally differentially private frequency estimation. JRR divides data contributors into disjoint groups of two members and lets those in the same group jointly perturb their binary data to improve frequency-estimation accuracy and achieve the same level of data privacy by hiding the group membership information in contrast to the classical RR mechanism. Theoretical analysis and detailed simulation studies using both real and synthetic datasets show that JRR achieves the same level of data privacy as the classical RR mechanism while improving the frequency-estimation accuracy in the overwhelming majority of the cases by up to two orders of magnitude.

In a decentralized Internet of Things (IoT) network, a fusion center receives information from multiple sensors to infer a public hypothesis of interest. To prevent the fusion center from abusing the sensor information, each sensor sanitizes its local observation using a local privacy mapping, which is designed to achieve both inference privacy of a private hypothesis and data privacy of the sensor raw observations. Various inference and data privacy metrics have been proposed in the literature. We introduce the concept of privacy implication (with vanishing budget) to study the relationships between these privacy metrics. We propose an optimization framework in which both local differential privacy (data privacy) and information privacy (inference privacy) metrics are incorporated. In the parametric case where sensor observations' distributions are known a priori, we propose a two-stage local privacy mapping at each sensor, and show that such an architecture is able to achieve information privacy and local differential privacy to within the predefined budgets. For the nonparametric case where sensor distributions are unknown, we adopt an empirical optimization approach. Simulation and experiment results demonstrate that our proposed approaches allow the fusion center to accurately infer the public hypothesis while protecting both inference and data privacy.

Local differential privacy is one of the most effective methods for privacy protection data publishing, whose theoretical basis is the randomised response. However, the existing models assume that data attributes are independent of each other, which might result in excessive information loss. To solve this issue, we present a local differentially private method for releasing association data. First, to find the relationship between attributes, we constructed a Bayesian network with a greedy algorithm base on mutual information for the given dataset. Second, to ensure local differential privacy, we perturbed each dependent attribute pair according to weak or robust association attribute set. Third, to achieve the local differential privacy with the noisy marginal, we constructed an approximation distribution for the given dataset. Finally, we experimentally evaluated our method on real data, and the extensive results show that our method better balances data utility and privacy disclosure.

To protect privacy for data-collection-based services, local differential privacy (LDP) is widely adopted due to its rigorous theoretical bound on privacy loss.However, mistakes in complex theoretical analysis or subtle implementation errors may undermine its practical guarantee.To address this, auditing is crucial to confirm that LDP protocols truly protect user data.However, existing auditing methods, though, mainly target machine learning and federated learning tasks based on centralized differentially privacy (DP), with limited attention to LDP.Moreover, the few studies on LDP auditing focus solely on simple frequency estimation task for discrete data, leaving correlated key-value data -which requires both discrete frequency estimation for keys and continuous mean estimation for values -unexplored.To bridge this gap, we propose KV-Auditor, a framework for auditing LDP-based key-value estimation mechanisms by estimating their empirical privacy lower bounds.Rather than traditional LDP auditing methods that relies on binary output predictions, KV-Auditor estimates this lower bound by analyzing unbounded output distributions, supporting continuous data.Specifically, we classify state-of-the-art LDP key-value mechanisms into interactive and non-interactive types.For non-interactive mechanisms, we propose horizontal KV-Auditor for small domains with sufficient samples and vertical KV-Auditor for large domains with limited samples.For interactive mechanisms, we design a segmentation strategy to capture incremental privacy leakage across iterations.Finally, we perform extensive experiments to validate the effectiveness of our approach, offering insights for optimizing LDP-based key-value estimators. CCS Concepts Security and privacy Data anonymization and sanitiza-

Frequency estimation mechanisms are widely applied in domains such as machine learning and cloud computing, where it is desirable to provide statistical information. As a fundamental operation in these domains, frequency estimation utilizes personal data which contains sensitive information while it is necessary to protect sensitive information from others. Motivated by this, we preserve user’s privacy with local differential privacy by obfuscating personal data on the user side. In this paper, we propose frequency estimation mechanisms under utility-optimized local differential privacy (ULDP), which allow the data collector to obtain some non-sensitive values to improve data utility while protecting sensitive values from leaking sensitive information. We propose three frequency estimation mechanisms under <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$(\epsilon ,\delta )$</tex-math></inline-formula> -ULDP (uRFM-GRR, uRFM-RAPPOR, uRFM-OLH) to preserve user’s sensitive information. Our proposed mechanisms protect sensitive data with the same privacy guarantee and they are suitable for different scenarios. Besides, in theory, we compare the estimation errors of our proposed mechanisms with existing LDP based mechanisms and show that ours are lower than theirs. Finally, we conduct experiments on synthetic and real-world datasets to evaluate the performance of the three mechanisms. The experimental results demonstrate that our proposed mechanisms are better than the existing LDP based solutions over the same privacy level, while uRFM-OLH frequently performs the best.

With the rapid development of information technologies, more and more data are collected from multiple sources, which contain different perspectives of the data. To accurately explore the shared information among multiple views, K-means based multi-view clustering methods are designed and widely used in various applications for their simplicity and efficiency. However, all of these methods cluster data in the original high-dimensional feature space which is extremely time-consuming and sensitive to outliers, or cluster data in the embedded feature space for each view, which is hard to find the optimal reduced dimensionality. To solve these problems, we propose a robust discriminative multi-view K-means clustering with feature selection and group sparsity learning. Compared to the state-of-the-arts, the proposed algorithm has two advantages: 1) Discriminative K-means clustering and feature learning are integrated jointly into a single framework, where robust and accurate clustering results are obtained in the embedded feature space with an l2, 1-norm based loss function. 2) Group sparsity constraints are imposed to select the most relevant features and the most important views. We apply the proposed algorithm to serval kinds of multimedia understanding applications. Experimental results demonstrate the effectiveness of the proposed algorithm.

Clustering algorithm is one of the most important algorithms in unsupervised learning. For density‐based spatial clustering of applications with noise (DBSCAN) density clustering algorithm, the selection of neighborhood radius and minimum number is the key to get the best clustering results. Aiming at the problems of traditional DBSCAN algorithm, such as the neighborhood radius and the minimum number of points, this article puts forward two classifications based on K‐means algorithm, and gets two clustering centers. Where calculated between two data points and the cluster center‐to ‐center distance, clustering, distance, statistics in a distance of data points within the scope of the search, the number of data points corresponding to the maximum distance value, and thus the parameters for the DBSCAN algorithm to estimate and selection of initial radius of neighborhood with the minimum number of clustering start critical value. When the parameters are iterated and optimized continuously, the data are divided into clusters, and the most suitable neighborhood radius and the minimum point number are obtained. The experimental data analysis show that the improved algorithm reduces the human factors in the traditional algorithm and improves the efficiency, so as to get the accurate clustering results.

Evaluation of the performance of clustering algorithms in kernel-induced feature space

Personal data privacy is a great concern by governments across the world as citizens generate huge amount of data continuously and industries using this for betterment of user centric services. There must be a reasonable balance between data privacy and utility of data. Differential privacy is a promise by data collector to the customer’s personal privacy. Centralised Differential Privacy (CDP) is performing output perturbation of user’s data by applying required privacy budget. This promises the inclusion or exclusion of individual’s data in data set not going to create significant change for a statistical query output and it offers -Differential privacy guarantee. CDP is holding a strong belief on trusted data collector and applying global sensitivity of the data. Local Differential Privacy (LDP) helps user to locally perturb his data and there by guaranteeing privacy even with untrusted data collector. Many differential privacy algorithms handles parameters like privacy budget, sensitivity and data utility in different ways and mostly trying to keep trade-off between privacy and utility of data. This paper evaluates differential privacy algorithms in regard to the privacy support it offers according to the sensitivity of the data. Generalized application of privacy budget is found ineffective in comparison to the sensitivity based usage of privacy budget.

Estimating spatial distributions is important in data analysis, such as traffic flow forecasting and epidemic prevention. To achieve accurate spatial distribution estimation, the analysis needs to collect sufficient user data. However, collecting data directly from individuals could compromise their privacy. Most previous works focused on private distribution estimation for one-dimensional data, which does not consider spatial data relation and leads to poor accuracy for spatial distribution estimation. In this paper, we address the problem of private spatial distribution estimation, where we collect spatial data from individuals and aim to minimize the distance between the actual distribution and estimated one under Local Differential Privacy (LDP). To leverage the numerical nature of the domain, we project spatial data and its relationships onto a one-dimensional distribution. We then use this projection to estimate the overall spatial distribution. Specifically, we propose a reporting mechanism called Disk Area Mechanism (DAM), which projects the spatial domain onto a line and optimizes the estimation using the sliced Wasserstein distance. Through extensive experiments, we show the effectiveness of our DAM approach on both real and synthetic data sets, compared with the state-of-the-art methods, such as Multi-dimensional Square Wave Mechanism (MDSW) and Subset Exponential Mechanism with Geo-I (SEM-Geo-I). Our results show that our DAM always performs better than MDSW and is better than SEM-Geo-I when the data granularity is fine enough.

Ensemble Clustering has been developed to provide an alternative way of obtaining more stable and accurate clustering results. It aims to avoid the biases of individual clustering algorithms. However, it is still a challenge to develop an efficient and robust method for Ensemble Clustering. Based on an existing ensemble clustering method, Consensus Clustering (CC), this paper introduces an advanced Consensus Clustering algorithm called Multi-Optimisation Consensus Clustering (MOCC), which utilises an optimised Agreement Separation criterion and a Multi-Optimisation framework to improve the performance of CC. Fifteen different data sets are used for evaluating the performance of MOCC. The results reveal that MOCC can generate more accurate clustering results than the original CC algorithm.KeywordsConsensus ClusteringSimulated AnnealingMulti-optimisation