Improving Frequency Estimation under Local Differential Privacy

Federated learning (FL) allows to train a massive amount of data privately due to its decentralized structure. Stochastic gradient descent (SGD) is commonly used for FL due to its good empirical performance, but sensitive user information can still be inferred from weight updates shared during FL iterations. We consider Gaussian mechanisms to preserve local differential privacy (LDP) of user data in the FL model with SGD. The trade-offs between user privacy, global utility, and transmission rate are proved by defining appropriate metrics for FL with LDP. Compared to existing results, the query sensitivity used in LDP is defined as a variable, and a tighter privacy accounting method is applied. The proposed utility bound allows heterogeneous parameters over all users. Our bounds characterize how much utility decreases and transmission rate increases if a stronger privacy regime is targeted. Furthermore, given a target privacy level, our results guarantee a significantly larger utility and a smaller transmission rate as compared to existing privacy accounting methods.

With the advent of the era of big data, privacy issues have been becoming a hot topic in public. Local differential privacy (LDP) is a state-of-the-art privacy preservation technique that allows to perform big data analysis (e.g., statistical estimation, statistical learning, and data mining) while guaranteeing each individual participant’s privacy. In this paper, we present a comprehensive survey of LDP. We first give an overview on the fundamental knowledge of LDP and its frameworks. We then introduce the mainstream privatization mechanisms and methods in detail from the perspective of frequency oracle and give insights into recent studied on private basic statistical estimation (e.g., frequency estimation and mean estimation) and complex statistical estimation (e.g., multivariate distribution estimation and private estimation over complex data) under LDP. Furthermore, we present current research circumstances on LDP including the private statistical learning/inferencing, private statistical data analysis, privacy amplification techniques for LDP, and some application fields under LDP. Finally, we identify future research directions and open challenges for LDP. This survey can serve as a good reference source for the research of LDP to deal with various privacy-related scenarios to be encountered in practice.

Local differential privacy (LDP) provides a way for an untrusted data collector to aggregate users' data without violating their privacy. Various privacy-preserving data analysis tasks have been studied under the protection of LDP, such as frequency estimation, frequent itemset mining, and machine learning. Despite its privacy-preserving properties, recent research has demonstrated the vulnerability of certain LDP protocols to data poisoning attacks. However, existing data poisoning attacks are focused on basic statistics under LDP, such as frequency estimation and mean/variance estimation. As an important data analysis task, the security of LDP frequent itemset mining has yet to be thoroughly examined. In this paper, we aim to address this issue by presenting novel and practical data poisoning attacks against LDP frequent itemset mining protocols. By introducing a unified attack framework with composable attack operations, our data poisoning attack can successfully manipulate the state-of-the-art LDP frequent itemset mining protocols and has the potential to be adapted to other protocols with similar structures. We conduct extensive experiments on three datasets to compare the proposed attack with four baseline attacks. The results demonstrate the severity of the threat and the effectiveness of the proposed attack.

Local Differential Privacy (LDP) has been widely recognized as a powerful tool for providing a strong theoretical guarantee of data privacy to data contributors against an untrusted data collector. Under a typical LDP scheme, each data contributor independently randomly perturbs their data before submitting them to the data collector, which in turn infers valuable statistics about the original data from received perturbed data. Common to existing LDP mechanisms is an inherent trade-off between the level of privacy protection and data utility in the sense that strong data privacy often comes at the cost of reduced data utility. Frequency estimation based on Randomized Response (RR) is a fundamental building block of many LDP mechanisms. In this paper, we propose a novel Joint Randomized Response (JRR) mechanism based on correlated data perturbations to achieve locally differentially private frequency estimation. JRR divides data contributors into disjoint groups of two members and lets those in the same group jointly perturb their binary data to improve frequency-estimation accuracy and achieve the same level of data privacy by hiding the group membership information in contrast to the classical RR mechanism. Theoretical analysis and detailed simulation studies using both real and synthetic datasets show that JRR achieves the same level of data privacy as the classical RR mechanism while improving the frequency-estimation accuracy in the overwhelming majority of the cases by up to two orders of magnitude.

Frequent itemset mining is a basic data mining task and has many applications in other data mining tasks. However, it is likely that the user's personal privacy information may be leaked in the mining process. In recent years, applying the local differential privacy protection model to mine all frequent itemsets is a relatively reliable and secure protection method. In local differential privacy, users first perturb the original data then send it to the aggregator, which prevents the aggregator leaking user's private information throughout the process. There are two major problems with data mining using local differential privacy, one is that the accuracy of the results is relatively low after mining, and the other is that the user transmits too much data to the server, which results in higher communication costs. In this paper, we use Hadamard response algorithm to improve the accuracy of the results while reduce the communication cost. Finally, we use FP-tree for frequent itemset mining to compare the Hadamard response with previous algorithms.

SummaryIn big data mining, the K‐prototypes has become a popular clustering method for mixed data owing to its simplicity and efficiency. However, the data clustering process of the K‐prototypes method will cause the risk of user privacy leakage because user data usually contain sensitive information. To address this issue, general solutions introduce a trusted third‐party model for privacy protection in clustering analysis, but it is difficult to find a fully trusted entity in reality. In this article, we propose a local differential privacy K‐prototypes (LDPK) mechanism, which does not require any trusted third party to perform privacy preprocessing on user data. Our mechanism first uses local differential privacy to disturb user data, then completes the clustering through the interaction between server and user. Furthermore, we propose a privacy protection enhancement mechanism (ELDPK) by extending the LDPK mechanism, which disturbs the user's clustering information in each iteration to protect user privacy further. Theoretical analysis proves the privacy and feasibility of our proposed scheme, and the experimental results prove that our proposed scheme can guarantee the quality of the clustering results under the premise of satisfying the local differential privacy.

Personal data privacy is a great concern by governments across the world as citizens generate huge amount of data continuously and industries using this for betterment of user centric services. There must be a reasonable balance between data privacy and utility of data. Differential privacy is a promise by data collector to the customer’s personal privacy. Centralised Differential Privacy (CDP) is performing output perturbation of user’s data by applying required privacy budget. This promises the inclusion or exclusion of individual’s data in data set not going to create significant change for a statistical query output and it offers -Differential privacy guarantee. CDP is holding a strong belief on trusted data collector and applying global sensitivity of the data. Local Differential Privacy (LDP) helps user to locally perturb his data and there by guaranteeing privacy even with untrusted data collector. Many differential privacy algorithms handles parameters like privacy budget, sensitivity and data utility in different ways and mostly trying to keep trade-off between privacy and utility of data. This paper evaluates differential privacy algorithms in regard to the privacy support it offers according to the sensitivity of the data. Generalized application of privacy budget is found ineffective in comparison to the sensitivity based usage of privacy budget.

Local differential privacy (LDP) is a promising privacy model for data collection that protects sensitive information of individuals. However, applying LDP to top-<inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> estimation in set-valued data (e.g., identifying most frequent <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> items) may yield poor results for small and sparse datasets due to high sensitivity and heavy perturbation. To address this, we propose an adaptive approach that frames the problem as a multi-armed bandit (MAB) problem, in which the decision-maker selects actions based on information collected from previous rounds to maximize the total reward over time. Inspired by this, we present two adaptive sampling schemes based on MAB: <b>ARBS</b> for identifying top-<inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> items and <b>ARBSF</b> for both top-<inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> item discovery and frequency estimation on these items. Furthermore, to address the potential long delay of multi-round collection, we propose an optimization technique to reduce the time complexity. Both theoretical and empirical results show that our adaptive sampling schemes significantly outperform existing alternatives.

Local Differential Privacy (LDP) provides a formal guarantee of privacy that enables the collection and analysis of sensitive data without revealing any individual's data. While LDP methods have been extensively studied, there is a lack of a systematic and empirical comparison of LDP methods for descriptive statistics. In this paper, we first provide a systematization of LDP methods for descriptive statistics, comparing their properties and requirements. We demonstrate that several mean estimation methods based on sampling from a Bernoulli distribution are equivalent in the one-dimensional case and introduce methods for variance estimation. We then empirically compare methods for mean, variance, and frequency estimation. Finally, we provide recommendations for the use of LDP methods for descriptive statistics and discuss their limitations and open questions.

Local differential privacy federated learning has attracted wide attention because it can solve the problem of data islands without damaging data privacy, but it faces data heterogeneity problems on the client side. Existing solutions commonly cluster clients with similar data distributions into the same training groups by leveraging clients’ model parameters, thus mitigating the impact of data heterogeneity on federated learning performance. However, model parameters, as an indirect representation for client data, often fail to accurately reflect the true data distribution, resulting in inaccurate client groupings. Furthermore, these solutions perturb high-dimensional parameter vectors dimension-by-dimension to protect client data privacy, which introduces substantial LDP noise that significantly further compromises the client grouping accuracy. To tackle these challenges, we propose PCFed-LDP, a privacy-preserving clustered federated learning framework that improves the federated learning performance while protecting client data and satisfying LDP in heterogeneous environments. Specifically, we introduce a client clustering method based on geometric properties of client data subspaces, which conducts label grouping-based principal angle analysis on client data subspaces to accurately capture the similarities in client data distributions, thereby enabling precise client grouping. To reduce the amount of introduced LDP noise, we design an adaptive noise addition method that utilizes the Haar wavelet technique to decouple the relationship between the noise amount and vector dimensionality, and employs noise error minimization strategy-based vector segmentation to inject LDP noise with finer granularity. Theoretical analysis and experiments on real datasets demonstrate that our solution not only satisfies the constraints of local differential privacy but also outperforms state-of-the-art methods.

In local differential privacy (LDP), each user perturbs her data locally before sending the noisy data to a data collector. The latter then analyzes the data to obtain useful statistics. Unlike the setting of centralized differential privacy, in LDP the data collector never gains access to the exact values of sensitive data, which protects not only the privacy of data contributors but also the collector itself against the risk of potential data leakage. Existing LDP solutions in the literature are mostly limited to the case that each user possesses a tuple of numeric or categorical values, and the data collector computes basic statistics such as counts or mean values. To the best of our knowledge, no existing work tackles more complex data mining tasks such as heavy hitter discovery over set-valued data. In this paper, we present a systematic study of heavy hitter mining under LDP. We first review existing solutions, extend them to the heavy hitter estimation, and explain why their effectiveness is limited. We then propose LDPMiner, a two-phase mechanism for obtaining accurate heavy hitters with LDP. The main idea is to first gather a candidate set of heavy hitters using a portion of the privacy budget, and focus the remaining budget on refining the candidate set in a second phase, which is much more efficient budget-wise than obtaining the heavy hitters directly from the whole dataset. We provide both in-depth theoretical analysis and extensive experiments to compare LDPMiner against adaptations of previous solutions. The results show that LDPMiner significantly improves over existing methods. More importantly, LDPMiner successfully identifies the majority true heavy hitters in practical settings.

In a decentralized Internet of Things (IoT) network, a fusion center receives information from multiple sensors to infer a public hypothesis of interest. To prevent the fusion center from abusing the sensor information, each sensor sanitizes its local observation using a local privacy mapping, which is designed to achieve both inference privacy of a private hypothesis and data privacy of the sensor raw observations. Various inference and data privacy metrics have been proposed in the literature. We introduce the concept of privacy implication (with vanishing budget) to study the relationships between these privacy metrics. We propose an optimization framework in which both local differential privacy (data privacy) and information privacy (inference privacy) metrics are incorporated. In the parametric case where sensor observations' distributions are known a priori, we propose a two-stage local privacy mapping at each sensor, and show that such an architecture is able to achieve information privacy and local differential privacy to within the predefined budgets. For the nonparametric case where sensor distributions are unknown, we adopt an empirical optimization approach. Simulation and experiment results demonstrate that our proposed approaches allow the fusion center to accurately infer the public hypothesis while protecting both inference and data privacy.

To protect privacy for data-collection-based services, local differential privacy (LDP) is widely adopted due to its rigorous theoretical bound on privacy loss.However, mistakes in complex theoretical analysis or subtle implementation errors may undermine its practical guarantee.To address this, auditing is crucial to confirm that LDP protocols truly protect user data.However, existing auditing methods, though, mainly target machine learning and federated learning tasks based on centralized differentially privacy (DP), with limited attention to LDP.Moreover, the few studies on LDP auditing focus solely on simple frequency estimation task for discrete data, leaving correlated key-value data -which requires both discrete frequency estimation for keys and continuous mean estimation for values -unexplored.To bridge this gap, we propose KV-Auditor, a framework for auditing LDP-based key-value estimation mechanisms by estimating their empirical privacy lower bounds.Rather than traditional LDP auditing methods that relies on binary output predictions, KV-Auditor estimates this lower bound by analyzing unbounded output distributions, supporting continuous data.Specifically, we classify state-of-the-art LDP key-value mechanisms into interactive and non-interactive types.For non-interactive mechanisms, we propose horizontal KV-Auditor for small domains with sufficient samples and vertical KV-Auditor for large domains with limited samples.For interactive mechanisms, we design a segmentation strategy to capture incremental privacy leakage across iterations.Finally, we perform extensive experiments to validate the effectiveness of our approach, offering insights for optimizing LDP-based key-value estimators. CCS Concepts Security and privacy Data anonymization and sanitiza-

Nowadays, people pay more and more attention to personal privacy issues, and how to collect data while protecting personal privacy and data security has become a major challenge. Local differential privacy, as a new privacy protection model, can collect and protect user data without a trusted third party. Among the many local differential privacy algorithms, RAPPOR(Randomized Aggregatable Privacy-Preserving Ordinal Response) is suitable for situations where the privacy budget is small. However, RAPPOR uses Bloom filters to map the data, which can lead to the same encoding result for different data, and the existing solution is to group the data. The paper, we propose the digital RAPPOR algorithm D-RAPPOR (Digitization-RAPPOR) does not require grouping of user data, D-RAPPOR is based on user data on the ordinal number of candidate values The D-RAPPOR algorithm is suitable for data sets with known candidate values. In this paper, the D-RAPPOR algorithm is used for experimental analysis of electricity data, which can be collected safely. The method can obtain the individual frequencies of the customer's electricity data while ensuring the security of the customer's electricity data. The control center can analyze the results of the algorithm to achieve the purpose of regulating electricity. Finally, the algorithm proposed in this paper solves the problem that the original algorithm produces the same encoding, and the results of the D-RAPPOR algorithm are slightly better than the RAPPOR algorithm under the same privacy budget.

While real-time electricity consumption data of users in smart grids can enable value-added services, such as Big Data analytics, each individual user's privacy needs to be protected. How to balance data utility and privacy protection is a significant issue, of which privacy-preserving data aggregation (PPDA) is a viable solution. Prior to this, researchers have proposed a number of PPDA schemes to address the above challenge. Unluckily, most of them suffer from security and privacy drawbacks, while others are inappropriate for resource-limited smart meters due to high cost of cryptographic operations. To tackle this issue, in this article, we propose a pairing-free and exponentiation-free certificateless PPDA scheme named CL-PPDA for smart grids. We prove the security of our design and analyze its performance. Comparative analyses with state-of-the-art work in theory and experiment show that our design not only has better security properties, but also has competitive computation overhead, especially on the resource-constrained smart meter side. Besides, we present an extension of our CL-PPDA scheme to support multidimensional data aggregation, which further enriches the functionality of our design.