PIGEON: A High Throughput Framework for Private Inference of Neural Networks using Secure Multiparty Computation

Deep neural networks (DNNs) are typically optimized using various forms of mini-batch gradient descent algorithm. A major motivation for mini-batch gradient descent is that with a suitably chosen batch size, available computing resources can be optimally utilized (including parallelization) for fast model training. However, many works report the progressive loss of model generalization when the training batch size is increased beyond some limits. This is a scenario commonly referred to as generalization gap. Although several works have proposed different methods for alleviating the generalization gap problem, a unanimous account for understanding generalization gap is still lacking in the literature. This is especially important given that recent works have observed that several proposed solutions for generalization gap problem such learning rate scaling and increased training budget do not indeed resolve it. As such, our main exposition in this paper is to investigate and provide new perspectives for the source of generalization loss for DNNs trained with a large batch size. Our analysis suggests that large training batch size results in increased near-rank loss of units’ activation (i.e. output) tensors, which consequently impacts model optimization and generalization. Extensive experiments are performed for validation on popular DNN models such as VGG-16, residual network (ResNet-56) and LeNet-5 using CIFAR-10, CIFAR-100, Fashion-MNIST and MNIST datasets.

The widespread use of Batch Normalization has enabled training deeper neural networks with more stable and faster results. However, the Batch Normalization works best using large batch size during training and as the state-of-the-art segmentation convolutional neural network architectures are very memory demanding, large batch size is often impossible to achieve on current hardware. We evaluate the alternative normalization methods proposed to solve this issue on a problem of binary spine segmentation from 3D CT scan. Our results show the effectiveness of Instance Normalization in the limited batch size neural network training environment. Out of all the compared methods the Instance Normalization achieved the highest result with Dice coefficient = 0.96 which is comparable to our previous results achieved by deeper network with longer training time. We also show that the Instance Normalization implementation used in this experiment is computational time efficient when compared to the network without any normalization method.

In international nuclear safeguards, the International Atomic Energy Agency (IAEA) is tasked with inspecting and verifying nuclear facilities and their activities. Data analytics and machine learning to support inspections require large amounts of data that nuclear facility operators may consider proprietary or sensitive, so the IAEA may not have full access. Allowing computation over private data without compromising its security therefore has value for safeguards inspections and analysis. Privacy-preserving machine learning (PPML) consists of security-focused techniques that allow data analytics and machine learning algorithms to run on sensitive data without revealing it. This includes ideas like homomorphic encryption (HE), secure multiparty computation (SMPC), and secure enclaves. HE allows algorithms and mathematical operations to be conducted directly on the encrypted data instead of first decrypting it. With SMPC, multiple entities collaboratively compute over distributed data such that no party is able to directly view any others’ original data. Secure enclaves allow computation to take place in a separate and heavily blocked-off section of a CPU. Techniques like these allow for several potential use cases in which the security of data is essential. With SMPC, machine learning models can be trained over the input data from multiple entities, resulting in a model that all users can benefit from without leaking the input data from any particular entity. With SMPC or a zero-knowledge proof (ZKP), an algorithm returning some single answer or truth value can be run on someone else’s data without ever needing to see that data, potentially allowing for verification or proof of some underlying question. HE can allow for outsourcing computation on data to a hostile or untrusted environment. Although most of the research in this field resides within the health and financial domains, tools from PPML may have similar applications in nuclear safeguards. Allowing the IAEA to compute over proprietary information, such as process models and raw sensor data using PPML techniques, provides the baseline for running complex analytics without needing direct unencrypted access to the underlying data, maintaining its privacy. Important limitations to consider for these techniques include the efficiency and level of security required. The security of HE and SMPC come at the cost of speed—the significant amount of overhead means that algorithms implemented in these protocols and encryption schemes are slower than when run on plaintext. Additionally, several important parameters determine what techniques or protocols are used based on the security requirements. SMPC protocols may need to be selected for resistance against a party that attempts to deviate from the protocol to distort the result or gain access to additional information, and a protocol secure against these attacks may further increase the overhead of the algorithm.

Secure multi-party computations for privacy-preserving machine learning

Mixed-protocol multi-party computation framework towards complex computation tasks with malicious security

BackgroundGenome-wide association studies (GWAS) aim to identify genetic variants (usually single nucleotide polymorphisms [SNPs]) across the entire human genome that are associated with phenotypic traits such as disease status and drug response. Highly accurate and reproducible genotype calling are paramount since errors introduced by calling algorithms can lead to inflation of false associations between genotype and phenotype. Most genotype calling algorithms currently used for GWAS are based on multiple arrays. Because hundreds of gigabytes (GB) of raw data are generated from a GWAS, the samples are typically partitioned into batches containing subsets of the entire dataset for genotype calling. High call rates and accuracies have been achieved. However, the effects of batch size (i.e., number of chips analyzed together) and of batch composition (i.e., the choice of chips in a batch) on call rate and accuracy as well as the propagation of the effects into significantly associated SNPs identified have not been investigated. In this paper, we analyzed both the batch size and batch composition for effects on the genotype calling algorithm BRLMM using raw data of 270 HapMap samples analyzed with the Affymetrix Human Mapping 500 K array set.ResultsUsing data from 270 HapMap samples interrogated with the Affymetrix Human Mapping 500 K array set, three different batch sizes and three different batch compositions were used for genotyping using the BRLMM algorithm. Comparative analysis of the calling results and the corresponding lists of significant SNPs identified through association analysis revealed that both batch size and composition affected genotype calling results and significantly associated SNPs. Batch size and batch composition effects were more severe on samples and SNPs with lower call rates than ones with higher call rates, and on heterozygous genotype calls compared to homozygous genotype calls.ConclusionBatch size and composition affect the genotype calling results in GWAS using BRLMM. The larger the differences in batch sizes, the larger the effect. The more homogenous the samples in the batches, the more consistent the genotype calls. The inconsistency propagates to the lists of significantly associated SNPs identified in downstream association analysis. Thus, uniform and large batch sizes should be used to make genotype calls for GWAS. In addition, samples of high homogeneity should be placed into the same batch.

Cross-modal recipe retrieval aims to exploit the relationships and accomplish mutual retrieval between recipe images and texts, which is clear for human but arduous to formulate. Although many previous works endeavored to solve this problem, most works did not efficiently exploit the cross-modal information among recipe data. In this paper, we present a frustratingly straightforward cross-modal recipe retrieval framework, Transformer-based Network for Large Batch Training (TNLBT) achieving high performance on both recipe retrieval and image generation tasks, which is designed to efficiently exploit the rich cross-modal information. In our proposed framework, Transformer-based encoders are applied for both image and text encoding for cross-modal embedding learning. We also adopt several loss functions like self-supervised learning loss on recipe text to encourage the model to further promote the cross-modal embedding learning. Since contrastive learning could benefit from a larger batch size according to the recent literature on self-supervised learning, we adopt a large batch size during training and have validated its effectiveness. The experimental results showed that TNLBT significantly outperformed the current state-of-the-art frameworks in both cross-modal recipe retrieval and image generation tasks on the benchmark Recipe1M by a huge margin. We also found that CLIP-ViT performs better than ViT-B as the image encoder backbone. This is the first work which confirmed the effectiveness of large batch training on cross-modal recipe embedding learning.

This paper explores real-time Convolutional Neural Network inference on Field Pro- grammable Gate Arrays (FPGAs) implemented in Synchronous Message Exchange (SME). We compare SME to the widespread FPGA tool, High-Level Synthesis (HLS), and com- pare both the SME and HLS implementations of CNNs with the PyTorch implementation for CNN on CPU/GPU. We find that the SME implementation is more flexible than the HLS implementation as it allows for more customization of the hardware. Programming with SME is more difficult than HLS, although easier than traditional Hardware Descrip- tion Languages. Finally, for a test use case, we find that the SME implementation on FPGA is approximately 2.8/1.4/2.0 times more energy efficient than CPU/GPU/ARM at larger batch sizes, with the HLS implementation on FPGA falling in between CPU/ARM and GPU in terms of energy efficiency. At a batch size of 1, appropriate for edge-device inference, the gap in energy efficiency between the FPGA and CPU/GPU/ARM imple- mentations becomes more pronounced, with the SME implementation on FPGA being approximately 83/47/8 times more energy efficient than the CPU/GPU/ARM implemen- tations, and with the HLS implementation on FPGA being approximately 40/23/4 times more energy efficient than the CPU/GPU/ARM implementations.

Deep learning architectures are usually proposed with millions of parameters, resulting in a memory issue when training deep neural networks with stochastic gradient descent type methods using large batch sizes. However, training with small batch sizes tends to produce low quality solution due to the large variance of stochastic gradients. In this paper, we tackle this problem by proposing a new framework for training deep neural network with small batches/noisy gradient. During optimization, our method iteratively applies a proximal type regularizer to make loss function strongly convex. Such regularizer stablizes the gradient, leading to better training performance. We prove that our algorithm achieves comparable convergence rate as vanilla SGD even with small batch size. Our framework is simple to implement and can be potentially combined with many existing optimization algorithms. Empirical results show that our method outperforms SGD and Adam when batch size is small. Our implementation is available at https://github.com/huiqu18/TRAlgorithm.

Machine learning (ML) has revolutionized real-time decision-making, enabling significant advancements in fields such as healthcare, cybersecurity, and autonomous systems. Despite these strides, two critical challenges hinder broader adoption: the lack of transparency in complex models and concerns about preserving user privacy. This paper introduces a comprehensive framework that combines explainable artificial intelligence (XAI) with privacy-preserving machine learning (PPML), tailored for systems requiring real-time responsiveness. The framework employs innovative methods, including interpretable modeling, federated learning, and secure computation, to balance accuracy with ethical considerations. Rigorous evaluations on benchmark datasets, particularly in healthcare and energy optimization, highlight its effectiveness, with results demonstrating a 15% increase in interpretability scores and 20% enhancement in privacy adherence compared to existing approaches. By addressing these critical barriers, the proposed framework establishes a foundation for integrating ML ethically and efficiently into real-time applications.

Over the past few years, machine learning has been responsible for the rapid advancements in fields such as computer vision, natural language processing and speech recognition. No small part of this success is due to data becoming more and more available, often being collected in privacy-invasive ways. The aim of this work is to study the use of two privacy-preserving solutions for training machine learning models: Federated Learning (FL) and Secure Multiparty Computation (MPC). Federated learning is a subfield of machine learning that allows training models on a large, decentralized corpus of data residing on edge devices like smartphones. Instead of sharing data, users collaboratively train a model by only sending weight updates to a server. By leveraging secure multiparty computation, it can be ensured that the server cannot inspect any individual user's update. To assess the feasibility of these approaches in different settings, a client-server architecture was implemented in Python and multiple experiments were run on datasets made available by LEAF in order to investigate ways of improving the overall performance of the models trained in a federated manner.

Self-supervised pretraining has shown impressive performance in recent years, matching or even outperforming ImageNet weights on a broad range of downstream tasks. Unfortunately, existing methods require massive amounts of computing power with large batch sizes and batch norm statistics synchronized across multiple GPUs. This effectively excludes substantial parts of the computer vision community from the benefits of self-supervised learning who do not have access to extensive computing resources.To address that, we develop FastSiam with the aim of matching ImageNet weights given as little computing power as possible. We find that a core weakness of previous methods like SimSiam is that they compute the training target based on a single augmented crop (or “view”), leading to target instability. We show that by using multiple views per image instead of one, the training target can be stabilized, allowing for faster convergence and substantially reduced runtime. We evaluate FastSiam on multiple challenging downstream tasks including object detection, instance segmentation and keypoint detection and find that it matches ImageNet weights after 25 epochs of pretraining on a single GPU with a batch size of only 32.KeywordsResource-efficientSelf-supervised learning

Diffusion processes in semiconductor fabrication facilities (Fabs) refer to the series of processes from wafer cleaning processes to furnace processes. Most furnace tools are batch tools, with large batch sizes, and have relatively long process times, when compared to the other processes. Strict time window constraints link cleaning processes with furnace processes for quality control. Those operational requirements for diffusion processes make their scheduling very difficult. This paper proposes an advanced scheduling approach based on a rolling horizon scheduling concept. Due to the combinatorial nature of the scheduling problem, the complexity of the problem increases exponentially, when the number of jobs and tools increase. However, the computation time allowed for the scheduler is limited in practice, because the variability in most Fabs requires schedulers to update the schedule in short intervals. We suggest an mixed integer linear programming model for diffusion processes, and propose an effective decomposition method to deal with this complexity problem. The decomposition method repeats multiple scheduling iterations, as it gradually extends the number of runs on tools, enabling the scheduler to generate near-optimal schedules in limited time intervals. The scheduler could make large improvements on key performance indicators, such as time window violation rates, batch sizes, throughput, etc. The software architecture of the scheduler implementation is also addressed in this paper.

In modern distributed systems, efficient write operations are critical to achieving high performance and responsiveness. However, as systems scale, the overhead associated with managing writes can become a bottleneck, particularly when batching techniques are used. Write batching is a common strategy to reduce the overhead of frequent write operations by grouping them together into a single batch. This approach reduces the number of transactions or communication rounds required to commit data, thus improving throughput. However, it also introduces a significant challenge: high write latency. High write latency occurs when the time it takes to write data to the system becomes excessive, resulting in delayed responses to clients and reduced overall system efficiency. The latency in write batching systems is often influenced by several factors, including the time required to accumulate enough writes to form a full batch, the coordination required between nodes, and the delay in flushing the batch to persistent storage or replicating it across the system If there are many write requests arriving at the same time, the system might need to handle conflicts or lock resources, leading to delays before writes can be committed. In some cases, partial batches—where not enough writes have accumulated—may need to be handled separately, further adding to the overhead. The high write latency in batching systems becomes especially pronounced when dealing with large numbers of nodes or when the network communication is slow or unreliable. As the system scales horizontally, the coordination and communication overhead grow, potentially exacerbating the delay in committing batches. This trade-off between write efficiency and latency is a fundamental challenge in distributed systems. If the system has a large batch size or a long flush period, this waiting time can lead to a noticeable delay in processing individual write requests. To address this issue, optimizing write batching systems requires strategies that balance the need for efficient coordination with minimizing delays. Reducing quorum requirements, implementing more efficient conflict resolution mechanisms, and dynamically adjusting batch sizes can help mitigate the impact of high write latency. Furthermore, leveraging algorithms that reduce the number of communication rounds, such as optimizing consensus protocols, can also help achieve better performance while ensuring consistency across the system. This paper addresses the high write latency issue which write operations are facing with the help of fast paxos algorithm.

SUMMARY In this paper, we examine and quantify the effects of introducing new processing flexibility in the rapidly changing environment of printed circuit board (PCB) assembly. Specifically, we perform experiments designed to establish the benefits and determine the limitations associated with the introduction of routeing flexibility. We examine system behaviour as batch sizes are reduced in an environment characterized by moderate set-up requirements. We further demonstrate that the performance of flexible routeing policies in this environment is adversely affected by larger batch sizes and that the introduction of routeing flexibility leads to improved system performance in small batch assembly.