HomoPAI: A Secure Collaborative Machine Learning Platform based on Homomorphic Encryption

e18725 Background: Healthcare data sharing is important for the creation of diverse and large data sets, supporting clinical decision making, and accelerating efficient research to improve patient outcomes. This is especially vital in the case of real world data analysis. However, stakeholders are reluctant to share their data without ensuring patients’ privacy, proper protection of their data sets and the ways they are being used. Homomorphic encryption is a cryptographic capability that can address these issues by enabling computation on encrypted data without ever decrypting it, so the analytics results are obtained without revealing the raw data. The aim of this study is to prove the accuracy of analytics results and the practical efficiency of the technology. Methods: A real-world data set of colorectal cancer patients’ survival data, following two different treatment interventions, including 623 patients and 24 variables, amounting to 14,952 items of data, was encrypted using leveled homomorphic encryption implemented in the PALISADE software library. Statistical analysis of key oncological endpoints was blindly performed on both the raw data and the homomorphically-encrypted data using descriptive statistics and survival analysis with Kaplan-Meier curves. Results were then compared with an accuracy goal of two decimals. Results: The difference between the raw data and the homomorphically encrypted data results, regarding all variables analyzed was within the pre-determined accuracy range goal, as well as the practical efficiency of the encrypted computation measured by run time, are presented in table. Conclusions: This study demonstrates that data encrypted with Homomorphic Encryption can be statistical analyzed with a precision of at least two decimal places, allowing safe clinical conclusions drawing while preserving patients’ privacy and protecting data owners’ data assets. Homomorphic encryption allows performing efficient computation on encrypted data non-interactively and without requiring decryption during computation time. Utilizing the technology will empower large-scale cross-institution and cross- stakeholder collaboration, allowing safe international collaborations. Clinical trial information: 0048-19-TLV. [Table: see text]

With the wide application of cloud computing in multi-tenant, heterogeneous nodes and high-concurrency environments, model parameters frequently interact during distributed training, which easily leads to privacy leakage, communication redundancy, and decreased aggregation efficiency. To realize the collaborative optimization of privacy protection and computing performance, this study proposes the Heterogeneous Federated Homomorphic Encryption Cloud (HFHE-Cloud) model, which integrates federated learning (FL) and homomorphic encryption and constructs a secure and efficient collaborative learning framework for cloud platforms. Under the condition of not exposing the original data, the model effectively reduces the performance bottleneck caused by encryption calculation and communication delay through hierarchical key mapping and dynamic scheduling mechanism of heterogeneous nodes. The experimental results show that HFHE-Cloud is significantly superior to Federated Averaging (FedAvg), Federated Proximal (FedProx), Federated Personalization (FedPer) and Federated Normalized Averaging (FedNova) in comprehensive performance, Homomorphically Encrypted Federated Averaging (HE-FedAvg) and other five baseline models. In the dimension of privacy protection, the global accuracy is up to 94.25%, and the Loss is stable within 0.09. In terms of computing performance, the encryption and decryption time is shortened by about one third, and the encryption overhead is controlled at 13%. In terms of distributed training efficiency, the number of communication rounds is reduced by about one fifth, and the node participation rate is stable at over 90%. The results verify the model's ability to achieve high security and high scalability in multi-tenant environment. This study aims to provide cloud service providers and enterprise data holders with a technical solution of high-intensity privacy protection and efficient collaborative training that can be deployed in real cloud platforms.

Machine-Learning-as-a-Service has become increasingly popular, with Recommendation-as-a-Service as one of the representative examples. In such services, providing privacy protection for the users is an important topic. Reviewing privacy-preserving solutions which were proposed in the past decade, privacy and machine learning are often seen as two competing goals at stake. Though improving cryptographic primitives (e.g., secure multi-party computation (SMC) or homomorphic encryption (HE)) or devising sophisticated secure protocols has made a remarkable achievement, but in conjunction with state-of-the-art recommender systems often yields far-from-practical solutions. We tackle this problem from the direction of machine learning. We aim to design crypto-friendly recommendation algorithms, thus to obtain efficient solutions by directly using existing cryptographic tools. In particular, we propose an HE-friendly recommender system, refer to as CryptoRec, which (1) decouples user features from latent feature space, avoiding training the recommendation model on encrypted data; (2) only relies on addition and multiplication operations, making the model straightforwardly compatible with HE schemes. The properties turn recommendation-computations into a simple matrix-multiplication operation. To further improve efficiency, we introduce a sparse-quantization-reuse method which reduces the recommendation-computation time by $9\times$ (compared to using CryptoRec directly), without compromising the accuracy. We demonstrate the efficiency and accuracy of CryptoRec on three real-world datasets. CryptoRec allows a server to estimate a user's preferences on thousands of items within a few seconds on a single PC, with the user's data homomorphically encrypted, while its prediction accuracy is still competitive with state-of-the-art recommender systems computing over clear data. Our solution enables Recommendation-as-a-Service on large datasets in a nearly real-time (seconds) level.

Histograms are widely used for summarizing data distributions, detecting anomalies, and improving machine learning models’ accuracy. However, traditional histogram-based methods require access to raw data, raising privacy concerns, particularly in sensitive IoT applications. Encryption-based techniques offer potential solutions; however, they secure the data in transit or storage, requiring decryption during analysis, which exposes raw data to potential privacy risks. In this paper, we propose a method for constructing privacy-preserving histograms directly on homomorphically encrypted IoT data, leveraging the Fast Fully Homomorphic Encryption over the Torus (TFHE) scheme implemented in the Concrete framework. To overcome the challenges posed by homomorphic encryption, we redesign the traditional histogram construction algorithm, optimizing it for secure computation by addressing constraints related to nested loops and conditional statements. As an evaluation use case, we have considered an outlier detection mechanism based on histogram frequency counts, ensuring that all data and computations remain encrypted throughout the process. Our method achieves results consistent with plaintext-based outlier detection while maintaining reasonable computational overhead compared to those reported in the existing literature.

Fully homomorphic encryption allows cloud servers to evaluate any computable functions for clients without revealing any information. It attracts much attention from both of the scientific community and the industry since Gentry’s seminal scheme. Currently, the Brakerski-Gentry-Vaikuntanathan scheme with its optimizations is one of the most potentially practical schemes and has been implemented in a homomorphic encryption C++ library HElib. HElib supplies friendly interfaces for arithmetic operations of polynomials over finite fields. Based on HElib, Chen and Guang (2015) implemented arithmetic over encrypted integers. In this paper, we revisit the HElib-based implementation of homomorphically arithmetic operations on encrypted integers. Due to several optimizations and more suitable arithmetic circuits for homomorphic encryption evaluation, our implementation is able to homomorphically evaluate 64-bit addition/subtraction and 16-bit multiplication for encrypted integers without bootstrapping. Experiments show that our implementation outperforms Chen and Guang’s significantly.

When users store their data on a cloud, they may concern on whether their data is stored correctly and can be fully retrieved. Proofs of Retrivability (PoR) is a cryptographic concept that allows users to remotely check the integrity of their data without downloading. This check is usually done by attaching data with message authenticators that contain data integrity information. The existing PoR schemes consider only the retrievability of unencrypted data and their message authenticators are usually deterministic. In this paper, we propose a PoR scheme that is built over homomorphic encryption schemes. Our PoR scheme can prove the retrievability of homomorphically encrypted data by generating probabilistic and homomorphic message authenticators. Moreover, the homomorphically encrypted data can be processed by the cloud directly and our PoR scheme can verify the integrity of such outsourced computations over ciphertexts. A prototype of our scheme is implemented to evaluate its performance.

While homomorphic encryption (HE) provides strong privacy protection, its high computational cost has restricted its application to simple tasks. Recently, hyperdimensional computing (HDC) applied to HE has shown promising performance for privacy-preserving machine learning (PPML). However, when applied to more realistic scenarios such as batch inference, the HDC-based HE has still very high compute time as well as high encryption and data transmission overheads. To address this problem, we propose HDC with encrypted parameters (EP-HDC), which is a novel PPML approach featuring client-side HE, i.e., inference is performed on a client using a homomorphically encrypted model. Our EP-HDC can effectively mitigate the encryption and data transmission overhead, as well as providing high scalability with many clients while providing strong protection for user data and model parameters. In addition to application examples for our client-side PPML, we also present design space exploration involving quantization, architecture, and HE-related parameters. Our experimental results using the BFV scheme and the Face/Emotion datasets demonstrate that our method can improve throughput and latency of batch inference by orders of magnitude over previous PPML methods ($36.52 \sim 1068 \times$ and $6.45 \sim 733 \times$, respectively) with <1% accuracy degradation.

Secure computation, a methodology of computing on encrypted data, has become a key factor in machine learning. Homomorphic encryption (HE) enables computation on encrypted data without leaking any information to untrusted servers. In machine learning, the model selection method is a crucial algorithm that determines the performance and reduces the fitting problem. Despite the importance of finding the optimal model, none of the previous studies have considered model selection when performing data analysis through the HE scheme. The HE-based model selection we proposed finds the optimal complexity that best describes given data that is encrypted and whose distribution is unknown. Since this process requires a matrix calculation, we constructed the matrix multiplication and inverse of the matrix based on the bitwise operation. Based on these, we designed the model selection of the HE cross-validation approach and the HE Bayesian approach for homomorphic machine learning. Our focus was on evidence approximation for linear models to find goodness-of-fit that maximizes the evidence. We conducted an experiment on a dataset of age and Body Mass Index (BMI) from Kaggle to compare the capabilities and our model showed that encrypted data can regress homomorphically without decrypting it.

ReActHE: A homomorphic encryption friendly deep neural network for privacy-preserving biomedical prediction

Although the emergence of massive data facilitates the applications of machine learning (ML) in precise biomedical-related prediction, there are growing concerns about the privacy and security of not only data but also ML models. It is still of urgent need to develop strategies to preserve privacy and protect the information security of data and models. Homomorphic encryption (HE) is one strategy that protects privacy of data and ML models by enabling direct computation over encrypted data, yet the computation burden of HE hinders its applications and the pre-processing involved in previous HE-based tasks poses the model to be vulnerable to the risk of membership inference attack. In this regard, we propose an integrated algorithm named Membership Inference Perturbation (MIP) - Sparse Homomorphic Encryption (SparseHE) method for privacy-preserving phenotype prediction. We design a novel sparse structure to reduce the computation redundancy of HE interface while being able to defend against membership inference attack via membership inference perturbation. We reduce the computation consumption by deploying a novel sparse homomorphic encryption interface on MIP structure. The theoretical results show that our SparseHE enables privacy-preserving healthcare prediction by tightening the generalization gap of model. Additionally, the proposed SparseHE accelerates the HE interface by reducing the computation redundancy. Our experimental results on four real healthcare datasets demonstrate that SparseHE can improve the ability of model to defend against MIA while reducing computation cost of HE prediction.

FedHEONN: Federated and homomorphically encrypted learning method for one-layer neural networks

In international nuclear safeguards, the International Atomic Energy Agency (IAEA) is tasked with inspecting and verifying nuclear facilities and their activities. Data analytics and machine learning to support inspections require large amounts of data that nuclear facility operators may consider proprietary or sensitive, so the IAEA may not have full access. Allowing computation over private data without compromising its security therefore has value for safeguards inspections and analysis. Privacy-preserving machine learning (PPML) consists of security-focused techniques that allow data analytics and machine learning algorithms to run on sensitive data without revealing it. This includes ideas like homomorphic encryption (HE), secure multiparty computation (SMPC), and secure enclaves. HE allows algorithms and mathematical operations to be conducted directly on the encrypted data instead of first decrypting it. With SMPC, multiple entities collaboratively compute over distributed data such that no party is able to directly view any others’ original data. Secure enclaves allow computation to take place in a separate and heavily blocked-off section of a CPU. Techniques like these allow for several potential use cases in which the security of data is essential. With SMPC, machine learning models can be trained over the input data from multiple entities, resulting in a model that all users can benefit from without leaking the input data from any particular entity. With SMPC or a zero-knowledge proof (ZKP), an algorithm returning some single answer or truth value can be run on someone else’s data without ever needing to see that data, potentially allowing for verification or proof of some underlying question. HE can allow for outsourcing computation on data to a hostile or untrusted environment. Although most of the research in this field resides within the health and financial domains, tools from PPML may have similar applications in nuclear safeguards. Allowing the IAEA to compute over proprietary information, such as process models and raw sensor data using PPML techniques, provides the baseline for running complex analytics without needing direct unencrypted access to the underlying data, maintaining its privacy. Important limitations to consider for these techniques include the efficiency and level of security required. The security of HE and SMPC come at the cost of speed—the significant amount of overhead means that algorithms implemented in these protocols and encryption schemes are slower than when run on plaintext. Additionally, several important parameters determine what techniques or protocols are used based on the security requirements. SMPC protocols may need to be selected for resistance against a party that attempts to deviate from the protocol to distort the result or gain access to additional information, and a protocol secure against these attacks may further increase the overhead of the algorithm.

Medical wearable devices monitor health data and, coupled with data analytics, cloud computing, and artificial intelligence (AI), enable early detection of disease. Privacy issues arise when personal health information is sent or processed outside the device. We propose a framework that ensures the privacy and integrity of personal medical data while performing AI-based homomorphically encrypted data analytics in the cloud. The main contributions are: (i) a privacy-preserving cloud-based machine learning framework for wearable devices, (ii) CipherML—a library for fast implementation and deployment of deep learning-based solutions on homomorphically encrypted data, and (iii) a proof-of-concept study for atrial fibrillation (AF) detection from electrocardiograms recorded on a wearable device. In the context of AF detection, two approaches are considered: a multi-layer perceptron (MLP) which receives as input the ECG features computed and encrypted on the wearable device, and an end-to-end deep convolutional neural network (1D-CNN), which receives as input the encrypted raw ECG data. The CNN model achieves a lower mean F1-score than the hand-crafted feature-based model. This illustrates the benefit of hand-crafted features over deep convolutional neural networks, especially in a setting with a small training data. Compared to state-of-the-art results, the two privacy-preserving approaches lead, with reasonable computational overhead, to slightly lower, but still similar results: the small performance drop is caused by limitations related to the use of homomorphically encrypted data instead of plaintext data. The findings highlight the potential of the proposed framework to enhance the functionality of wearables through privacy-preserving AI, by providing, within a reasonable amount of time, results equivalent to those achieved without privacy enhancing mechanisms. While the chosen homomorphic encryption scheme prioritizes performance and utility, certain security shortcomings remain open for future development.

Algorithmic trading has proliferated the area of quantitative finance for already over a decade. The decisions are made without human intervention using the data provided by brokerage firms and exchanges. There is an emerging intermediate layer of financial players that are placed in between a broker and algorithmic traders. The role of these players is to aggregate market decisions from the algorithmic traders and send a final market order to a broker. In return, the quantitative analysts receive incentives proportional to the correctness of their predictions. In such a setup, the intermediate player - an aggregator - does not provide the market data in plaintext but encrypts it. Encrypting market data prevents quantitative analysts from trading on their own, as well as keeps valuable financial data private. This paper proposes an implementation of a popular trend-following indicator with two different homomorphic encryption libraries - SEAL and HEAAN - and compares it to the trading indicator implemented for plaintext. Then an attempt to implement a trading strategy is presented and analysed. The trading indicator implemented with SEAL and HEAAN is almost identical to that implemented on the plaintext, the percentage error is of 0.14916% and 0.00020% respectively. Despite many limitations that homomorphic encryption imposes on this algorithm's implementation, quantitative finance has a high potential of benefiting from the methods of homomorphic encryption.

Machine learning servers with mass storage and computing power is an ideal platform to store, manage, and analyze data and support decision-making. However, the main issue is providing security and privacy to the data, as the data is stored in a public way. Recently, homomorphic data encryption has been proposed as a solution due to its capabilities in performing computations over encrypted data. In this paper, we proposed an encrypted all convolutional net that transformed traditional all convolutional net into a net based on homomorphic encryption. This scheme allows different data holders to send their encrypted data to cloud service, complete predictions, and return them in encrypted form as the cloud service provider does not have a secret key. Therefore, the cloud service provider and others cannot get unencrypted raw data. When applied to the MNIST database, privacy-preserving all convolutional based on homomorphic encryption predict efficiently, accurately and with privacy protection.