Membership Inference Attacks Against Deep Learning Models via Logits Distribution

Membership inference (MI) attacks highlight a privacy weakness in present stochastic training methods for neural networks. It is not well understood, however, why they arise. Are they a natural consequence of imperfect generalization only? Which underlying causes should we address during training to mitigate these attacks? Towards answering such questions, we propose the first approach to explain MI attacks and their connection to generalization based on principled causal reasoning. We offer causal graphs that quantitatively explain the observed MI attack performance achieved for $6$ attack variants. We refute several prior non-quantitative hypotheses that over-simplify or over-estimate the influence of underlying causes, thereby failing to capture the complex interplay between several factors. Our causal models also show a new connection between generalization and MI attacks via their shared causal factors. Our causal models have high predictive power ($0.90$), i.e., their analytical predictions match with observations in unseen experiments often, which makes analysis via them a pragmatic alternative.

Deep Learning (DL) techniques allow ones to train models from a dataset to solve tasks. DL has attracted much interest given its fancy performance and potential market value, while security issues are amongst the most colossal concerns. However, the DL models may be prone to the membership inference attack, where an attacker determines whether a given sample is from the training dataset. Efforts have been made to hinder the attack but unfortunately, they may lead to a major overhead or impaired usability. In this article, we propose and implement DAMIA, leveraging Domain Adaptation (DA) as a defense aginist membership inference attacks. Our observation is that during the training process, DA obfuscates the dataset to be protected using another relate and similar dataset, and derives a model that underlyingly extracts the features from both datasets. Seeing that the model is obfuscated, membership inference fails, while the extracted features provide supports for usability. Extensive experiments have been conducted to validates our intuition. The model trained by DAMIA has a negligible footprint to the usability and introduces slight overhead compared with other defenses. Our experiment also excludes factors that may hinder the performance of DAMIA, and comparisons with other defenses, providing a potential guideline to vendors and researchers to benefit from our solution in a timely manner.

Deep neural networks are susceptible to various inference attacks as they remember information about their training data. We design white-box inference attacks to perform a comprehensive privacy analysis of deep learning models. We measure the privacy leakage through parameters of fully trained models as well as the parameter updates of models during training. We design inference algorithms for both centralized and federated learning, with respect to passive and active inference attackers, and assuming different adversary prior knowledge. We evaluate our novel white-box membership inference attacks against deep learning algorithms to trace their training data records. We show that a straightforward extension of the known black-box attacks to the white-box setting (through analyzing the outputs of activation functions) is ineffective. We therefore design new algorithms tailored to the white-box setting by exploiting the privacy vulnerabilities of the stochastic gradient descent algorithm, which is the algorithm used to train deep neural networks. We investigate the reasons why deep learning models may leak information about their training data. We then show that even well-generalized models are significantly susceptible to white-box membership inference attacks, by analyzing state-of-the-art pre-trained and publicly available models for the CIFAR dataset. We also show how adversarial participants, in the federated learning setting, can successfully run active membership inference attacks against other participants, even when the global model achieves high prediction accuracies.

While person Re-identification (Re-ID) has progressed rapidly due to its wide real-world applications, it also causes severe risks of leaking personal information from training data. Thus, this paper focuses on quantifying this risk by membership inference (MI) attack. Most of the existing MI attack algorithms focus on classification models, while Re-ID follows a totally different training and inference paradigm. Re-ID is a fine-grained recognition task with complex feature embedding, and model outputs commonly used by existing MI like logits and losses are not accessible during inference. Since Re-ID focuses on modelling the relative relationship between image pairs instead of individual semantics, we conduct a formal and empirical analysis which validates that the distribution shift of the inter-sample similarity between training and test set is a critical criterion for Re-ID membership inference. As a result, we propose a novel membership inference attack method based on the inter-sample similarity distribution. Specifically, a set of anchor images are sampled to represent the similarity distribution conditioned on a target image, and a neural network with a novel anchor selection module is proposed to predict the membership of the target image. Our experiments validate the effectiveness of the proposed approach on both the Re-ID task and conventional classification task.

Machine learning models face increasing threats from membership inference attacks, which aim to infer sample membership. Sample membership represents whether or not a particular data sample is included in the training set of a given model and is considered a fundamental form of privacy leakage. Recent research has focused on the likelihood ratio attack, a membership inference attack that aggregates membership-relevant features through difficulty calibration and infers sample membership through hypothesis testing. However, difficulty calibration approaches typically require large amounts of labeled data to train shadow models, limiting their general applicability. Moreover, hypothesis testing often fails to identify training samples with low self-influence, resulting in suboptimal attack performance. To address these shortcomings, we propose Distilled Shadow Model and Inference Model to perform Membership Inference Attack (DSMIM-MIA) a novel membership inference attack that reduces the reliance on ground-truth labels through knowledge distillation and mitigates the bias against low self-influence samples using an inference model. Specifically, we distill the target model to train shadow models, which not only remove the dependence on labeled data but also transfer potential membership-relevant information to improve feature aggregation. In place of hypothesis testing, we train a neural network, referred to as the inference model, to predict sample membership. By learning membership decision functions directly from data, without relying on predefined statistical assumptions, our method achieves more accurate and generalizable predictions, especially for samples with low self-influence. Extensive experiments across three datasets and four model architectures demonstrate that DSMIM-MIA consistently outperforms existing state-of-the-art attacks under various evaluation metrics.

The arms race between attacks and defenses for machine learning models has come to a forefront in recent years, in both the security community and the privacy community. However, one big limitation of previous research is that the security domain and the privacy domain have typically been considered separately. It is thus unclear whether the defense methods in one domain will have any unexpected impact on the other domain. In this paper, we take a step towards resolving this limitation by combining the two domains. In particular, we measure the success of membership inference attacks against six state-of-the-art defense methods that mitigate the risk of adversarial examples (i.e., evasion attacks). Membership inference attacks determine whether or not an individual data record has been part of a model's training set. The accuracy of such attacks reflects the information leakage of training algorithms about individual members of the training set. Adversarial defense methods against adversarial examples influence the model's decision boundaries such that model predictions remain unchanged for a small area around each input. However, this objective is optimized on training data. Thus, individual data records in the training set have a significant influence on robust models. This makes the models more vulnerable to inference attacks. To perform the membership inference attacks, we leverage the existing inference methods that exploit model predictions. We also propose two new inference methods that exploit structural properties of robust models on adversarially perturbed data. Our experimental evaluation demonstrates that compared with the natural training (undefended) approach, adversarial defense methods can indeed increase the target model's risk against membership inference attacks. When using adversarial defenses to train the robust models, the membership inference advantage increases by up to 4.5 times compared to the naturally undefended models. Beyond revealing the privacy risks of adversarial defenses, we further investigate the factors, such as model capacity, that influence the membership information leakage.

Conventional membership inference attacks usually require a large number of queries of the target model when training shadow models, and this task becomes extremely difficult when the number of queries is limited. Aiming at the problem of insufficient training data for shadow models due to the limited number of queries, we propose a membership inference attack method based on generative adversarial networks (GAN). First, we use generative adversarial networks to augment the samples obtained by a small number of queries to expand the training data of the model; Secondly, we use the improved CNN to obtain shadow models that have a higher degree of fitting on different target model structures; Finally, we evaluate the accuracy of the proposed algorithm on XgBoost, Logistic, and neural network models using public datasets MNIST and CIFAR10 in a black-box setting, and the results show that our model has an average attack accuracy of 62% and 83%, respectively. It can be seen that, compared with the existing research methods, our model can obtain better attack effects under the condition of significantly reducing the number of queries, which shows the feasibility of our proposed method in membership inference attacks.

With the enhanced sensing and computing power of mobile devices, emerging technologies and applications generate massive amounts of data at the edge network, assigning new requirements for improving the privacy and security. Federated learning signifies the advancement in protecting the privacy of intelligent IoT applications. However, recent researches reveal the inherent vulnerabilities of the federated learning that the adversary could obtain the private training data from the publicly shared gradients. Although model distillation is considered a state-of-the-art technique to solve the gradient leakage by hiding gradients, our experiments show that it still has the risk of membership information leakage at the client level. In this paper, we propose a novel client-based membership inference attack in federated distillation learning. Specifically, we first comprehensively analyze model distillation in defensive capabilities for deep gradient leakage. Then, by exploiting that the adversary can learn other participants’ model structure and behavior during federated distillation, we design membership inference attacks against other participants based on private models of malicious clients. Experimental results demonstrate the superiority of our proposed method in terms of efficiency in resolving deep leakage of gradients and high accuracy of membership inference attack in federated distillation learning.KeywordsFederated learningModel distillationMembership inference attack

Recently, machine learning has been frequently applied in multiple areas, like image classification and object recognition. However, some machine learning applications train their models on sensitive data that could leak personal information, leading to potential privacy issues. Many privacy attacks against deep learning models have been proposed. A membership inference attack was demonstrated to be a type of effective attack and a real threat to privacy in machine learning. Here, we trained three different convolutional neural networks for image classification with the CIFAR10 dataset and examined the effects of a membership inference attack on these models. We also explore different defense strategies, including differential privacy against membership inference attack. We showed that membership inference attack was more effective against overfitting models and defense strategies like differential privacy and regularization can significantly lower the attack accuracy.

Location data is frequently collected from populations and shared in aggregate form to guide policy and decision making. However, the prevalence of aggregated data also raises the privacy concern of membership inference attacks (MIAs). MIAs infer whether an individual's data contributed to the aggregate release. Although effective MIAs have been developed for aggregate location data, these require access to an extensive auxiliary dataset of individual traces over the same locations, which are collected from a similar population. This assumption is often impractical given common privacy practices surrounding location data. To measure the risk of an MIA performed by a realistic adversary, we develop the first Zero Auxiliary Knowledge (ZK) MIA on aggregate location data, which eliminates the need for an auxiliary dataset of real individual traces. Instead, we develop a novel synthetic approach, such that suitable synthetic traces are generated from the released aggregate. We also develop methods to correct for bias and noise, to show that our synthetic-based attack is still applicable when privacy mechanisms are applied prior to release. Using two large-scale location datasets, we demonstrate that our ZK MIA matches the state-of-the-art Knock-Knock (KK) MIA across a wide range of settings, including popular implementations of differential privacy (DP) and suppression of small counts. Furthermore, we show that ZK MIA remains highly effective even when the adversary only knows a small fraction (10%) of their target's location history. This demonstrates that effective MIAs can be performed by realistic adversaries, highlighting the need for strong DP protection.

Neural networks are susceptible to data inference attacks such as the membership inference attack, the adversarial model inversion attack and the attribute inference attack, where the attacker could infer useful information such as the membership, the reconstruction or the sensitive attributes of a data sample from the confidence scores predicted by the target classifier. In this paper, we propose a method, namely PURIFIER, to defend against membership inference attacks. It transforms the confidence score vectors predicted by the target classifier and makes purified confidence scores indistinguishable in individual shape, statistical distribution and prediction label between members and non-members. The experimental results show that PURIFIER helps defend membership inference attacks with high effectiveness and efficiency, outperforming previous defense methods, and also incurs negligible utility loss. Besides, our further experiments show that PURIFIER is also effective in defending adversarial model inversion attacks and attribute inference attacks. For example, the inversion error is raised about 4+ times on the Facescrub530 classifier, and the attribute inference accuracy drops significantly when PURIFIER is deployed in our experiment.

Membership inference attacks can infer whether a data sample exists in the training set of the target model based on limited adversary knowledge, which results in serious leakage of privacy. A large number of recent studies have shown that model overfitting is one of the main reasons why membership inference attacks can be executed successfully. Therefore, some classic methods to solve model overfitting are used to defend against membership inference attacks, such as dropout, spatial dropout, and differential privacy. However, it is difficult for these defense methods to achieve an available trade-off in defense success rate and model utility. In this paper, we focus on the impact of model training loss on model overfitting, and we design a Squeeze-Loss strategy to dynamically find the training loss that achieves the best balance between model utility and privacy. Extensive experimental results show that our strategy can limit the success rate of membership inference attacks to the level of random guesses with almost no loss of model utility, which always outperforms other defense methods.KeywordsMembership inference attackSqueeze training lossDeep learningData privacyDefense strategy

Membership inference attacks (MIAs) are important measures to evaluate potential risks of privacy leakage from machine learning (ML) models. State-of-the-art MIA defenses have achieved favorable privacy-utility trade-offs using knowledge distillation on split training datasets. However, such defenses increase computational costs as a large number of the ML models must be trained on the split datasets. In this study, we proposed a new MIA defense, called SEDMA, based on self-distillation using model aggregation to mitigate the MIAs, inspired by the model parameter averaging as used in federated learning. The key idea of SEDMA is to split the training dataset into several parts and aggregate multiple ML models trained on each split for self-distillation. The intuitive explanation of SEDMA is that model aggregation prevents model over-fitting by smoothing information related to the training data among the multiple ML models and preserving the model utility, such as in federated learning. Through our experiments on major benchmark datasets (Purchase100, Texas100, and CIFAR100), we show that SEDMA outperforms state-of-the-art MIA defenses in terms of membership privacy (MIA accuracy), model accuracy, and computational costs. Specifically, SEDMA incurs at most approximately 3 - 5% model accuracy drop, while achieving the lowest MIA accuracy in state-of-the-art empirical MIA defenses. For computational costs, SEDMA takes significantly less processing time than a defense with the state-of-the-art privacy-utility trade-offs in previous defenses. SEDMA achieves both favorable privacy-utility trade-offs and low computational costs.

Membership inference attack for beluga whales discrimination

With breakthroughs in high-resolution image generation, applications for disentangled generative adversarial networks (GANs) have attracted much attention. At the same time, the privacy issues associated with GAN models have been raising many concerns. Membership inference attacks (MIAs), where an adversary attempts to determine whether or not a sample has been used to train the victim model, are a major risk with GANs. In prior research, scholars have shown that successful MIAs can be mounted by leveraging overfit images. However, high-resolution images make the existing MIAs fail due to their complexity. And the nature of disentangled GANs is such that the attributes are overfitting, which means that, for an MIA to be successful, it must likely be based on overfitting attributes. Furthermore, given the empirical difficulties with obtaining independent and identically distributed (IID) candidate samples, choosing the non-trivial attributes of candidate samples as the target for exploring overfitting would be a more preferable choice. Hence, in this paper, we propose a series of attribute-based MIAs that considers both black-box and white-box settings. The attacks are performed on the generator, and the inferences are derived by overfitting the non-trivial attributes. Additionally, we put forward a novel perspective on model generalization and a possible defense by evaluating the overfitting status of each individual attribute. A series of empirical evaluations in both settings demonstrate that the attacks remain stable and successful when using non-IID candidate samples. Further experiments illustrate that each attribute exhibits a distinct overfitting status. Moreover, manually generalizing highly overfitting attributes significantly reduces the risk of privacy leaks.