Heavy Hitter Estimation over Set-Valued Data with Local Differential Privacy

Most user-generated data in online services are presented as set-valued data, e.g., visited website URLs, recently used Apps by a person, and etc. These data are of great value to service providers, but also bring privacy concerns if collected and analyzed directly. To tackle potential privacy threatens, local differential privacy (LDP) attracts increasing attention nowadays. However, existing approaches only provide sub-optimal error bound for set-valued data distribution estimation with LDP. Besides, it is computational expensive and communication expensive to use for high dimensional set-valued data, considering large domains in real scenarios. Thus, existing approaches are unpractical to use on resource-constrained user-side devices (e.g., smartphones and wearable devices). In this paper, we propose a utility-optimal and efficient set-valued data publication method (i.e., wheel mechanism ). On the user side, each user contributes only one numerical value to represent their privatized data. The computational complexity is O (min{ m log m , me ɛ }) and communication cost is O (log( me ɛ )) bits, while existing approaches usually depend on O ( d ) or O (log d ), where m is the number of items in the set-valued data ( m ≡ 1 for categorical data), d is the domain size (usually d ≫ m ) and ɛ is the privacy budget. On the server side, the estimator takes numerical values from users as input and derives an unbiased distribution estimation. Theoretical results show that estimation error bounds are improved from previously known [EQUATION] to the optimal rate [EQUATION]. Results on extensive experiments demonstrate that our proposed wheel mechanism is 3-100× faster than existing approaches, meanwhile has optimal statistical efficiency.

Set-valued data is extremely important and widely used in sensor technology and application. Recently, privacy protection for set-valued data under differential privacy (DP) has become a research hotspot. However, the DP model assumes that the data center is trustworthy, consequently, increasingly attention has been paid to the application of the local differential privacy model (LDP) for set-valued data. Constrained by the local differential privacy model, most methods randomly respond to the subset of set-valued data, and the data collector conducts statistics on the received data. There are two main problems with this kind of method: one is that the utility function used in the random response loses too much information; the other is that the privacy protection of the set-valued data category is usually ignored. To solve these problems, this paper proposes a set-valued data collection method (SetLDP) based on the category hierarchy under the local differential privacy model. The core idea is to first make a random response to the existence of the category, continue to disturb the item count if the category exists, and finally randomly respond to a candidate itemset based on the new utility function. Theory analysis and experimental results show that the SetLDP can not only preserve more information, but also protect the category private information in set-valued data.

Many real-world social networks are decentralized in nature, and the only way to analyze such a network is to collect local views of the social graph from individual participants. Since local views may contain sensitive information, it is often desirable to apply differential privacy in the data collection process, which provides strong and rigorous privacy guarantees. In many practical situations, the local view of a participant contains not only her own connections, but also those of her neighbors, which are private and sensitive for the neighbors, but not directly so for the participant herself. We call such information beyond direct connections an extended local view (ELV), and study two fundamental problems related to ELVs: first, how do we correctly enforce differential privacy for all participants in the presence of ELVs? Second, how can the data collector utilize ELVs to obtain accurate estimates of global graph properties? This paper points out that when collecting ELVs, it is insufficient to apply a straightforward adaptation of local differential privacy (LDP), a commonly used scheme in practice, to protect the privacy of all network participants. The main problem is that an adversarial data collector can accumulate private information on a specific victim from multiple neighbors of the victim; even though the data collected from each neighbor is perturbed under LDP, their aggregate can still violate the victim's privacy. To prevent this attack, we formulate a novel decentralized differential privacy (DDP) scheme, which requires that each participant consider not only her own privacy, but also that of her neighbors involved in her ELV. The stringent privacy requirement of DDP, however, makes it challenging to design an effective mechanism for data collection. Towards this goal, we design a novel multi-phase framework under DDP that enables an analyst to accurately estimate subgraph counts, an important property of social graphs. The main idea is that instead of collecting subgraph counts directly, which would require excessively noise, the analyst first asks individuals about their respective minimum noise scale, which is private information since it depends on the local graph structure, and, thus, must be performed under DDP. For some types of subgraphs, this process is applied recursively, i.e., the analyst asks about the necessary noise to be injected into the private information on the minimum local noise scale required to protect subgraph counts under DDP. As case studies, we instantiate the proposed framework for three common subgraph patterns: triangles, three-hop paths, and k-cliques. Extensive experiments using real data demonstrate that the proposed scheme leads to accurate estimates of global subgraph counts, whereas baseline solutions fail to obtain meaningful result utility.

Personal data privacy is a great concern by governments across the world as citizens generate huge amount of data continuously and industries using this for betterment of user centric services. There must be a reasonable balance between data privacy and utility of data. Differential privacy is a promise by data collector to the customer’s personal privacy. Centralised Differential Privacy (CDP) is performing output perturbation of user’s data by applying required privacy budget. This promises the inclusion or exclusion of individual’s data in data set not going to create significant change for a statistical query output and it offers -Differential privacy guarantee. CDP is holding a strong belief on trusted data collector and applying global sensitivity of the data. Local Differential Privacy (LDP) helps user to locally perturb his data and there by guaranteeing privacy even with untrusted data collector. Many differential privacy algorithms handles parameters like privacy budget, sensitivity and data utility in different ways and mostly trying to keep trade-off between privacy and utility of data. This paper evaluates differential privacy algorithms in regard to the privacy support it offers according to the sensitivity of the data. Generalized application of privacy budget is found ineffective in comparison to the sensitivity based usage of privacy budget.

Local differential privacy (LDP) is a promising privacy model for data collection that protects sensitive information of individuals. However, applying LDP to top-<inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> estimation in set-valued data (e.g., identifying most frequent <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> items) may yield poor results for small and sparse datasets due to high sensitivity and heavy perturbation. To address this, we propose an adaptive approach that frames the problem as a multi-armed bandit (MAB) problem, in which the decision-maker selects actions based on information collected from previous rounds to maximize the total reward over time. Inspired by this, we present two adaptive sampling schemes based on MAB: <b>ARBS</b> for identifying top-<inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> items and <b>ARBSF</b> for both top-<inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> item discovery and frequency estimation on these items. Furthermore, to address the potential long delay of multi-round collection, we propose an optimization technique to reduce the time complexity. Both theoretical and empirical results show that our adaptive sampling schemes significantly outperform existing alternatives.

When collecting information, local differential privacy (LDP) alleviates privacy concerns of users because their private information is randomized before being sent it to the central aggregator. LDP imposes large amount of noise as each user executes the randomization independently. To address this issue, recent work introduced an intermediate server with the assumption that this intermediate server does not collude with the aggregator. Under this assumption, less noise can be added to achieve the same privacy guarantee as LDP, thus improving utility for the data collection task. This paper investigates this multiple-party setting of LDP. We analyze the system model and identify potential adversaries. We then make two improvements: a new algorithm that achieves a better privacy-utility tradeoff; and a novel protocol that provides better protection against various attacks. Finally, we perform experiments to compare different methods and demonstrate the benefits of using our proposed method.

With the advent of the era of big data, privacy issues have been becoming a hot topic in public. Local differential privacy (LDP) is a state-of-the-art privacy preservation technique that allows to perform big data analysis (e.g., statistical estimation, statistical learning, and data mining) while guaranteeing each individual participant’s privacy. In this paper, we present a comprehensive survey of LDP. We first give an overview on the fundamental knowledge of LDP and its frameworks. We then introduce the mainstream privatization mechanisms and methods in detail from the perspective of frequency oracle and give insights into recent studied on private basic statistical estimation (e.g., frequency estimation and mean estimation) and complex statistical estimation (e.g., multivariate distribution estimation and private estimation over complex data) under LDP. Furthermore, we present current research circumstances on LDP including the private statistical learning/inferencing, private statistical data analysis, privacy amplification techniques for LDP, and some application fields under LDP. Finally, we identify future research directions and open challenges for LDP. This survey can serve as a good reference source for the research of LDP to deal with various privacy-related scenarios to be encountered in practice.

In this paper, we tackle the problem of answering multi-dimensional range queries under local differential privacy. There are three key technical challenges: capturing the correlations among attrib...

High-dimensional crowdsourced data is pervasive in crowdsensing systems and with the development of IoTs it can produce rich knowledge for the society. However, it also creates serious privacy threats for crowdsourcing participants. To mitigate the privacy concerns in crowdsensing systems, local differential privacy has been derived from the de facto standard of differential privacy in order to achieve strong privacy guaranteed in distributed systems. However, directly achieving local differential privacy on high-dimensional crowdsourced data may lead not only to a prohibitive computational burden but also low data utility. Therefore, in this paper, we propose a local private high-dimensional data publication scheme for crowdsensing systems. In particular, on the participants side, high-dimensional records are locally perturbed to protect privacy, while on the servers side, the probability distribution of original data is recovered by taking advantage of both the expectation maximization algorithm and the theory of the Bayesian network. Extensive experiments on real-world datasets demonstrated the effectiveness of the proposed scheme that can synthesize approximate datasets with local differential privacy.

In the context of the Internet of Things (IoT), the large-scale generation and collection of data can greatly improve the quality of service provided, but they also raise significant concerns about privacy breaches. However, existing privacy-preserving data collection solutions based on local differential privacy (LDP) often struggle to balance security and accuracy when handling composite data types. To address this challenge, in this paper, we propose CSKV, a high-precision and privacy-preserving key-value data collection scheme. Specifically, we first design a padding and sampling protocol to improve data utility. Then, we propose two randomized response mechanisms to safely perturb keys and values in a cohesive and segmented manner. After that, by leveraging the sampling protocol and key-value correlation perturbation, we demonstrate that CSKV can provide secondary privacy amplification. Detailed theoretical analysis verifies the security and effectiveness of CSKV. In addition, extensive performance evaluations are conducted on synthetic and real-world datasets, and the results indicate that our proposed scheme outperforms existing schemes in terms of hit rate and estimation variance.

To protect privacy for data-collection-based services, local differential privacy (LDP) is widely adopted due to its rigorous theoretical bound on privacy loss.However, mistakes in complex theoretical analysis or subtle implementation errors may undermine its practical guarantee.To address this, auditing is crucial to confirm that LDP protocols truly protect user data.However, existing auditing methods, though, mainly target machine learning and federated learning tasks based on centralized differentially privacy (DP), with limited attention to LDP.Moreover, the few studies on LDP auditing focus solely on simple frequency estimation task for discrete data, leaving correlated key-value data -which requires both discrete frequency estimation for keys and continuous mean estimation for values -unexplored.To bridge this gap, we propose KV-Auditor, a framework for auditing LDP-based key-value estimation mechanisms by estimating their empirical privacy lower bounds.Rather than traditional LDP auditing methods that relies on binary output predictions, KV-Auditor estimates this lower bound by analyzing unbounded output distributions, supporting continuous data.Specifically, we classify state-of-the-art LDP key-value mechanisms into interactive and non-interactive types.For non-interactive mechanisms, we propose horizontal KV-Auditor for small domains with sufficient samples and vertical KV-Auditor for large domains with limited samples.For interactive mechanisms, we design a segmentation strategy to capture incremental privacy leakage across iterations.Finally, we perform extensive experiments to validate the effectiveness of our approach, offering insights for optimizing LDP-based key-value estimators. CCS Concepts Security and privacy Data anonymization and sanitiza-

SummaryIn big data mining, the K‐prototypes has become a popular clustering method for mixed data owing to its simplicity and efficiency. However, the data clustering process of the K‐prototypes method will cause the risk of user privacy leakage because user data usually contain sensitive information. To address this issue, general solutions introduce a trusted third‐party model for privacy protection in clustering analysis, but it is difficult to find a fully trusted entity in reality. In this article, we propose a local differential privacy K‐prototypes (LDPK) mechanism, which does not require any trusted third party to perform privacy preprocessing on user data. Our mechanism first uses local differential privacy to disturb user data, then completes the clustering through the interaction between server and user. Furthermore, we propose a privacy protection enhancement mechanism (ELDPK) by extending the LDPK mechanism, which disturbs the user's clustering information in each iteration to protect user privacy further. Theoretical analysis proves the privacy and feasibility of our proposed scheme, and the experimental results prove that our proposed scheme can guarantee the quality of the clustering results under the premise of satisfying the local differential privacy.

Local differential privacy (LDP) provides a way for an untrusted data collector to aggregate users' data without violating their privacy. Various privacy-preserving data analysis tasks have been studied under the protection of LDP, such as frequency estimation, frequent itemset mining, and machine learning. Despite its privacy-preserving properties, recent research has demonstrated the vulnerability of certain LDP protocols to data poisoning attacks. However, existing data poisoning attacks are focused on basic statistics under LDP, such as frequency estimation and mean/variance estimation. As an important data analysis task, the security of LDP frequent itemset mining has yet to be thoroughly examined. In this paper, we aim to address this issue by presenting novel and practical data poisoning attacks against LDP frequent itemset mining protocols. By introducing a unified attack framework with composable attack operations, our data poisoning attack can successfully manipulate the state-of-the-art LDP frequent itemset mining protocols and has the potential to be adapted to other protocols with similar structures. We conduct extensive experiments on three datasets to compare the proposed attack with four baseline attacks. The results demonstrate the severity of the threat and the effectiveness of the proposed attack.

As a novel distributed machine learning approach, federated learning (FL) is proposed to train a global model while preserving data privacy. However, some studies manifest that adversaries can still recover private information from the gradients. Differential privacy (DP) is a rigorous mathematical tool to protect records in a database against leakage. It has been widely applied in FL by perturbing the gradients. Nevertheless, while using DP in FL, the convergence performance of the global model is inevitably degraded. In this paper, we implement a DP-based FL scheme, which achieves local DP (LDP) by adding well-designed Gaussian noise on the gradients before clients upload them to the server. After that, we propose two strategies to improve the convergence performance of the DP-based FL. Both methods are realized by modifying the local objective function to limit the effect of LDP noise on convergence without degrading the privacy protection level. We then provide the detailed framework which adopts the LDP scheme and two strategies. The framework on different machine learning models is tested by simulation results, which show that our framework can improve the convergence performance up to 40% faster under different noise compared with other DP-based FL. Finally, we show the theoretical convergence guarantee of our proposed framework by first presenting the expected decrease in the global loss function for one round of training and then providing the upper convergence bound after multiple communication rounds.

Privacy concerns hinder the sharing and utilization of trajectory data collected from Internet of Things (IoT) devices. While local differential privacy mitigates leakage risks by perturbing data on mobile devices, existing methods primarily focus on enhancing global-level statistical usability of perturbed trajectories (such as spatial distributions and mobility transition patterns), neglecting individual-level semantic usability (such as road network consistency and traffic rule compliance). The resulting trajectories frequently contain unrealistic behaviors that violate traffic rules, severely limiting their applicability. To overcome this challenge, we propose TPIS, a trajectory protection approach with individual semantic utility under local differential privacy. First, we design a semantic-enhanced hierarchical modeling method (SEHM) that leverages traffic rule semantic information and geospatial information to model the road network as a weighted graph, supported by an R-tree index for efficient coarse-grained and fine-grained trajectory matching. Second, we propose a cascaded perturbation method combining stochastic sampling guided by the hierarchical graph model (SHG) and multi-feature fusion perturbation mechanism based on δ-location set (MFPD). SHG strategically constrains the perturbation space to preserve global-level statistical usability while ensuring rigorous privacy, and MFPD incorporates a comprehensive trajectory difference metric and δ -location set definition to generate perturbed trajectories that maintain individual-level structural integrity of real-world movement patterns. TPIS efficiently balances privacy and utility and ensures high computational efficiency. Theoretical analysis confirms its differential privacy guarantee and upper bound on perturbation error. Experiments conducted on four real-world datasets demonstrate significant improvements over existing methods, with up to 8× higher utility and 14× greater computational efficiency.