Distributional Black-Box Model Inversion Attack With Multi-Agent Reinforcement Learning

Model Inversion (MI) attacks, which reconstruct the training dataset of neural networks, pose significant privacy concerns in machine learning. Recent MI attacks have managed to reconstruct realistic label-level private data, such as the general appearance of a target person from all training images labeled on him. Beyond label-level privacy, in this paper we show sample-level privacy, the private information of a single target sample, is also important but under-explored in the MI literature due to the limitations of existing evaluation metrics. To address this gap, this study introduces a novel metric tailored for training-sample analysis, namely, the Diversity and Distance Composite Score (DDCS), which evaluates the reconstruction fidelity of each training sample by encompassing various MI attack attributes. This, in turn, enhances the precision of sample-level privacy assessments. Leveraging DDCS as a new evaluative lens, we observe that many training samples remain resilient against even the most advanced MI attack. As such, we further propose a transfer learning framework that augments the generative capabilities of MI attackers through the integration of entropy loss and natural gradient descent. Extensive experiments verify the effectiveness of our framework on improving state-of-the-art MI attacks over various metrics including DDCS, coverage and FID. Finally, we demonstrate that DDCS can also be useful for MI defense, by identifying samples susceptible to MI attacks in an unsupervised manner.

By abusing access to a well-trained classifier, model inversion (MI) attacks pose a significant threat as they can recover the original training data, leading to privacy leakage. Previous studies mitigated MI attacks by imposing regularization to reduce the dependency between input features and outputs during classifier training, a strategy known as unilateral dependency optimization. However, this strategy contradicts the objective of minimizing the supervised classification loss, which inherently seeks to maximize the dependency between input features and outputs. Consequently, there is a trade-off between improving the model's robustness against MI attacks and maintaining its classification performance. To address this issue, we propose the bilateral dependency optimization strategy (BiDO), a dual-objective approach that minimizes the dependency between input features and latent representations, while simultaneously maximizing the dependency between latent representations and labels. BiDO is remarkable for its privacy-preserving capabilities. However, models trained with BiDO exhibit diminished capabilities in out-of-distribution (OOD) detection compared to models trained with standard classification supervision. Given the open-world nature of deep learning systems, this limitation could lead to significant security risks, as encountering OOD inputs-whose label spaces do not overlap with the in-distribution (ID) data used during training-is inevitable. To address this, we leverage readily available auxiliary OOD data to enhance the OOD detection performance of models trained with BiDO. This leads to the introduction of an upgraded framework, unknown-aware BiDO (BiDO+), which mitigates both privacy and security concerns. As a highlight, with comparable model utility, BiDO-HSIC+ reduces the FPR95 by 55.02% and enhances the AUCROC by 9.52% compared to BiDO-HSIC, while also providing superior MI robustness.

This paper focuses on the information leakage and secure encoding transmission in semantic communication systems. A two-path images feature-coupling coding transmission is proposed to improve the transmission efficiency and bandwidth utilization while ensuring the restoration quality of each image. In particular, the designed encoder first extracts the feature of each-path image and then performs superimposed encoding in the feature space. In order to separately restore the two images at the decoder without successive interference cancellation or other traditional multi-user estimation, we add the device vectors to train the encoder and decoder. Unlike embedding the device vectors into each image, we embed the device vectors in the feature space instead. Furthermore, we construct an inversion attack network based on generative adversarial networks, which can simulate a potential eavesdropper and use the received transmission signal to recover the original image. Therefore, we can evaluate the system’s defense capabilities against model inversion attacks. Moreover, we use Huawei Ascend Atlas 200I DK A2 development board to conduct hardware simulation of legal user communication with proposed feature-coupling coding and the interception of information by eavesdroppers. Experimental results show that the proposed feature-coupling coding has better Peak Signal-to-Noise Ratio performance compared to the data–coupling coding and effectively resists information leakage among different noise level.

Deep learning models have achieved great success in many real-world tasks such as image recognition, machine translation, and self-driving cars. A large amount of data are needed to train a model, and in many cases, the training data are private. Publishing or sharing a deep learning model trained on private datasets could pose privacy concerns. We study the model inversion attacks against deep learning models, which attempt to reconstruct the features of training data corresponding to a given class given access to the model. While deep learning with differential privacy is state-of-the-art for training privacy-preserving models, whether they can provide meaningful protection against model inversion attacks remains an open question. In this paper, we first improve the existing model inversion attacks (MIA) to successfully reconstruct training images from neural network based image recognition models. Then, we demonstrate that deep learning with the standard record-level differential privacy does not provide quantifiable protection against MIA. Subsequently, we propose class-level and subclass-level differential privacy and develop algorithms to provide a quantifiable privacy guarantee against MIA. Experiments on real datasets demonstrate that our proposed privacy notions and mechanisms can effectively defend against MIA while maintaining model accuracy.

Machine learning has a vast outreach in principal applications and uses large amount of data to train the models, prompting a viable and easy to use Machine Learning as a Service (MLaaS). This flexible paradigm however, could have immense privacy implications since the training data often contains sensitive features, and adversarial access to such models could pose a security risk. In adversarial attacks such as model inversion attack on a system used for face recognition, an adversary uses the output (target label) to reconstruct the input (image of the target individual from the training dataset). To avert such a vulnerability of the system, in this paper, we develop a novel approach of applying perceptual hash to parts of the given training images that leverages the functional mechanism of image hashing. The facial recognition system is then trained over this newly created dataset of perceptually hashed images and high classification accuracy is observed. Furthermore, we demonstrate a series of model inversion attacks emulating adversarial access that yield hashed images of target individuals instead of the original training dataset images; thereby preventing original image reconstruction and counteracting the inversion attack. Through rigorous empirical evaluations of applying the proposed formulation over real world dataset, we verify the effectiveness of our proposed framework in protecting the training image dataset and counteracting inversion attack.

Previous model inversion attacks in collaborative inference systems have only been demonstrated on simple models. They failed to produce convincing results on large models or large images. To improve the quality of the model inversion attack, two adaptive generative model inversion attacks are proposed. First, the distributional prior of the publicly available pre-trained Generative Adversarial Nets are used to guide the reconstruction process. Second, previous works only search the latent space of the pre-trained GAN. However, the pre-trained generative model and the target model are trained on different datasets. The distribution shifts of the datasets can lead to inevitable reconstruction errors. To handle the distribution shifts, the pre-trained generative model will be fine-tuned for target instances in the proposed adaptive generative model inversion attack. As a result, the proposed attacks can better handle the distribution shifts. And there is no need to train a separate image prior for each target model. The generators can be used to attack different models trained on different datasets within the same domain. Our extensive experiments demonstrate that the proposed attack achieves up to a 25.7% improvement in PSNR and a 26.4% improvement in SSIM compared to previous methods.

Many individuals are concerned about the governance of machine learning systems and the prevention of algorithmic harms. The EU's recent General Data Protection Regulation (GDPR) has been seen as a core tool for achieving better governance of this area. While the GDPR does apply to the use of models in some limited situations, most of its provisions relate to the governance of personal data, while models have traditionally been seen as intellectual property. We present recent work from the information security literature around ‘model inversion’ and ‘membership inference’ attacks, which indicates that the process of turning training data into machine-learned systems is not one way, and demonstrate how this could lead some models to be legally classified as personal data. Taking this as a probing experiment, we explore the different rights and obligations this would trigger and their utility, and posit future directions for algorithmic governance and regulation.This article is part of the theme issue ‘Governing artificial intelligence: ethical, legal, and technical opportunities and challenges’.

<span lang="EN-US">Edge artificial intelligence (Edge AI) involves the implementation of AI algorithms and models directly on local edge devices, such as sensors or internet of things (IoT) devices. This allows for immediate processing and analysis of data without the need for continuous dependence on cloud infrastructure. Concerns about privacy have grown importance in recent years for businesses looking to uphold end-user expectations and safeguard business models. Federated learning (FL) has emerged as a novel approach to enhance privacy. To improve generalization qualities, FL trains local models on local data. These models then collaborate to update a global model. Each edge device (like smartphones, IoT sensors, or autonomous vehicles) trains a local model on its own data. This local training helps in capturing data patterns specific to each device or node. Poisoning, backdoors, and generative adversarial network (GAN)-based attacks are currently the main security risk. Nevertheless, the biggest threat to FL’s privacy is from inference-based assaults such as model inversion attacks, differential privacy shortcomings and FL utilizes blockchain and cryptography technologies to improve privacy on edge devices. This paper presents a thorough examination of the current literature on this subject. <br /> In more detail, we study the background of FL and its different existing applications, types, privacy threats and its techniques for privacy preservation.</span>

Generative adversarial network (GAN)-based models possess superior capability of high-fidelity image synthesis. There are a wide range of semantically meaningful directions in the latent representation space of well-trained GANs, and the corresponding latent space walks are meaningful for semantic controllability in the synthesized images. To explore the underlying organization of a latent space, we propose an unsupervised Density-Preserving Latent Semantics Exploration model (DP-LaSE). The important latent directions are determined by maximizing the variations in intermediate features, while the correlation between the directions is minimized. Considering that latent codes are sampled from a prior distribution, we adopt a density-preserving regularization approach to ensure latent space walks are maintained in iso-density regions, since moving to a higher/lower density region tends to cause unexpected transformations. To further refine semantics-specific transformations, we perform subspace learning over intermediate feature channels, such that the transformations are limited to the most relevant subspaces. Extensive experiments on a variety of benchmark datasets demonstrate that DP-LaSE is able to discover interpretable latent space walks, and specific properties of synthesized images can thus be precisely controlled.

Federated learning provides a solution for data privacy protection, while enabling training over the local data samples, without exchanging them. However, it is far from practical and secure because data privacy is still vulnerable due to the well-studied attacks, e.g., membership inference attacks and model inversion attacks. In this paper, to further prevent data leakage against these attacks, we propose FL-PATE, a differentially private federated learning framework with knowledge transfer. Specifically, participants with sensitive data are grouped to train teacher models under federated learning settings, and the knowledge of teacher models is transferred to a publicly accessible student model for prediction via aggregating teacher models' outputs of public datasets. A modified client-level differential privacy mechanism is used to guarantee each participant's data privacy during the corresponding teacher model's training process. The proposed framework preserves participant's privacy against membership inference attacks and the differential privacy cost is fixed. The privacy analysis and experiments demonstrate that trained teacher and student models have an excellent performance in accuracy and robustness theoretically and empirically.

With the advances in 5G and IoT devices, industries are vastly adopting artificial intelligence (AI) techniques for improving classification and prediction-based services. However, the use of AI also raises concerns regarding data privacy and security that can be misused or leaked. Private AI was recently coined to address the data security issue by combining AI with encryption techniques, but existing studies have shown that model inversion attacks can be used to reverse engineer the images from model parameters. In this regard, we propose a federated learning and encryption-based private (FLEP) AI framework that provides two-tier security for data and model parameters in an Industrial IoT environment. We propose a three-layer encryption method for data security and provided a hypothetical method to secure the model parameters. Experimental results show that the proposed method achieves better encryption quality at the expense of slightly increased execution time. We also highlight several open issues and challenges regarding the FLEP AI framework's realization.

Conditional generative adversarial networks (cGANs) target at synthesizing diverse images given the input conditions and latent codes, but unfortunately, they usually suffer from the issue of mode collapse. To solve this issue, previous works [47], [22] mainly focused on encouraging the correlation between the latent codes and their generated images, while ignoring the relations between images generated from various latent codes. The recent MSGAN [27] tried to encourage the diversity of the generated image but only considers "negative" relations between the image pairs.In this paper, we propose a novel DivCo framework to properly constrain both "positive" and "negative" relations between the generated images specified in the latent space. To the best of our knowledge, this is the first attempt to use contrastive learning for diverse conditional image synthesis. A novel latent-augmented contrastive loss is introduced, which encourages images generated from adjacent latent codes to be similar and those generated from distinct latent codes to be dissimilar. The proposed latent-augmented contrastive loss is well compatible with various cGAN architectures. Extensive experiments demonstrate that the proposed DivCo can produce more diverse images than state-of-the-art methods without sacrificing visual quality in multiple unpaired and paired image generation tasks. Training code and pretrained models are available at https://github.com/ruiliu-ai/DivCo.

HSGAN: Reducing mode collapse in GANs by the latent code distance of homogeneous samples

The simultaneous pursuit of operational efficiency and strict data sovereignty presents a critical dilemma for modern urban logistics, particularly within the constrained infrastructure of developing economies. While Federated Learning (FL) offers a decentralized pathway to mitigate privacy risks, its direct application in Vehicle-to-Everything (V2X) networks is frequently compromised by heterogeneous data quality and prohibitive communication overhead. To resolve these limitations, this study proposes a Privacy-Preserving V2X Federated Learning (PPVFL) framework specifically engineered for sustainable last-mile delivery. Unlike conventional approaches that treat data quality and privacy as separate domains, the proposed method enforces a strict local outlier filtering protocol to sanitize traffic beacons at the source, integrated synergistically with sparse ternary compression and differential privacy. This unified architecture not only safeguards sensitive driver trajectories against model inversion attacks but also drastically reduces the bandwidth consumption required for global model aggregation. Empirical validation using real-world logistics data from Vietnam demonstrates that delegating quality control to the network edge enables the framework to outperform centralized and basic federated baselines in both prediction accuracy and energy efficiency. These findings articulate a scalable solution for green logistics that reconciles the trade-off between robust traffic modeling and compliance with stringent data protection standards.

With the wide application of machine learning technology, more and more sensitive data were used to develop the machine learning model. Unfortunately, machine learning systems have been shown to be vulnerable to various privacy leakage attacks, such as membership inference attacks and model inversion attacks. These privacy threats break down the confidentiality of the training data; what’s more, the leakage of sensitive privacy will reduce the enthusiasm of data owners for sharing data with the machine learning system. Without enough training data, more seriously, it will hinder the application of machine learning. Therefore, it requires analyzing strategies for privacy risks to help data owners pre-assess the candidate dataset and assist data owners in implementing reasonable privacy control. However, the systematic privacy risk assessment is still absent from the data owner’s perspective.This paper investigates and analyzes machine learning privacy risks to understand the relationship between training data properties and privacy leakage. Based on this recognition, we introduce a privacy risk assessment scheme based on the clustering distance of training data. Our clustering distance-based method can reflect the privacy risk level of a different individual data record. And then, we combine both existing privacy analysis based on data features and our clustering distance-based method to investigate privacy risks systematically. Our experiments showed that our clustering distance method and other set properties are tightly related to privacy leakage. And data owners can pre-assess the privacy risks of candidate datasets before uploading or sharing them to the machine learning model. If needed, decide to re-choose the dataset to reduce privacy risks to an acceptable level.