ReActHE: A homomorphic encryption friendly deep neural network for privacy-preserving biomedical prediction

Homomorphic encryption (HE) has emerged as a pivotal technology for secure neural network inference (SNNI), offering privacy-preserving computations on encrypted data. Despite active developments in this field, HE-based SNNI frameworks are impeded by three inherent limitations. Firstly, they cannot evaluate non-linear functions such as ReLU, the most widely adopted activation function in neural networks. Secondly, the permitted number of homomorphic operations on ciphertexts is bounded, consequently limiting the depth of neural networks that can be evaluated. Thirdly, the computational overhead associated with HE is prohibitively high, particularly for deep neural networks. In this paper, we introduce a novel paradigm designed to address the three limitations of HE-based SNNI. Our approach is an interactive approach that is solely based on HE, called iLHE. Utilizing the idea of iLHE, we present two protocols: ReLU, which facilitates the direct evaluation of the ReLU function on encrypted data, tackling the first limitation, and HeRefresh, which extends the feasible depth of neural network computations and mitigates the computational overhead, thereby addressing the second and third limitations. Based on HeReLU and HeRefresh protocols, we build a new framework for SNNI, named HeFUN. We prove that our protocols and the HeFUN framework are secure in the semi-honest security model. Empirical evaluations demonstrate that HeFUN surpasses current HE-based SNNI frameworks in multiple aspects, including security, accuracy, the number of communication rounds, and inference latency. Specifically, for a convolutional neural network with four layers on the MNIST dataset, HeFUN achieves 99.16% accuracy with an inference latency of 1.501 s, surpassing the popular HE-based framework CryptoNets proposed by Gilad-Bachrach, which achieves 98.52% accuracy with an inference latency of 3.479 s.

HEFactory: A symbolic execution compiler for privacy-preserving Deep Learning with Homomorphic Encryption

Homomorphic Encryption (HE) allows encrypted data to be processed without decryption, which could maximize the protection of user privacy without affecting the data utility. Thanks to strides made by cryptographers in the past few years, the efficiency of HE has been drastically improved, and machine learning on homomorphically encrypted data has become possible. Several works have explored machine learning based on HE, but most of them are restricted to the outsourced scenario, where all the data comes from a single data owner. We propose HomoPAI, an HE-based secure collaborative machine learning system, enabling a more promising scenario, where data from multiple data owners could be securely processed. Moreover, we integrate our system with the popular MPI framework to achieve parallel HE computations. Experiments show that our system can train a logistic regression model on millions of homomorphically encrypted data in less than two minutes.

Deep learning algorithms can be used to design systems that report data on patients and deliver warnings to medical applications or electronic health information if there are changes in the patient's health. These systems could be created using deep Learning. This helps verify that patients get the proper effective care at the proper time for each specific patient. A healthcare decision support system was presented using the Internet of Things and deep learning methods. In the proposed system, we examined the capability of integrating deep learning technology into automatic diagnosis and IoT capabilities for faster message exchange over the Internet. We have selected the suitable Neural Network structure (number of best-hidden layers and activation function classes) to construct the e-health system. In addition, the e-health system relied on data from doctors to understand the Neural Network. In the validation method, the total evaluation of the proposed healthcare system for diagnostics provides dependability under various patient conditions. Based on evaluation and simulation findings, a dual hidden layer of feed-forward NN and its neurons store the tanh function more effectively than other NN. To overcome challenges, this study will integrate artificial intelligence with IoT. This study aims to determine the NN's optimal layer counts and activation function variations.

Homomorphic encryption (HE) permits users to perform computations on encrypted data without first decrypting it. HE can be used for privacy-preserving outsourced computation and analysis, allowing data to be encrypted and outsourced to commercial cloud environments for processing while encrypted or sensitive data. HE enables new services by removing privacy barriers inhibiting data sharing or increasing the security of existing services. A convolution neural network (CNN) can be homomorphically evaluated using addition and multiplication by replacing the activation function, such as Rectified Linear Units (ReLU), with a low polynomial degree. To achieve the same performance as the ReLU activation function, we study the impact of applying the ensemble techniques to solve the accuracy problem. Our experimental results empirically show that the ensemble approach can reduce bias, and variance, increasing accuracy to achieve the same ReLU performance with parallel and sequential techniques. We demonstrate the effectiveness and robustness of our method using three datasets: MNIST, FMNIST, and CIFAR-10.

e18725 Background: Healthcare data sharing is important for the creation of diverse and large data sets, supporting clinical decision making, and accelerating efficient research to improve patient outcomes. This is especially vital in the case of real world data analysis. However, stakeholders are reluctant to share their data without ensuring patients’ privacy, proper protection of their data sets and the ways they are being used. Homomorphic encryption is a cryptographic capability that can address these issues by enabling computation on encrypted data without ever decrypting it, so the analytics results are obtained without revealing the raw data. The aim of this study is to prove the accuracy of analytics results and the practical efficiency of the technology. Methods: A real-world data set of colorectal cancer patients’ survival data, following two different treatment interventions, including 623 patients and 24 variables, amounting to 14,952 items of data, was encrypted using leveled homomorphic encryption implemented in the PALISADE software library. Statistical analysis of key oncological endpoints was blindly performed on both the raw data and the homomorphically-encrypted data using descriptive statistics and survival analysis with Kaplan-Meier curves. Results were then compared with an accuracy goal of two decimals. Results: The difference between the raw data and the homomorphically encrypted data results, regarding all variables analyzed was within the pre-determined accuracy range goal, as well as the practical efficiency of the encrypted computation measured by run time, are presented in table. Conclusions: This study demonstrates that data encrypted with Homomorphic Encryption can be statistical analyzed with a precision of at least two decimal places, allowing safe clinical conclusions drawing while preserving patients’ privacy and protecting data owners’ data assets. Homomorphic encryption allows performing efficient computation on encrypted data non-interactively and without requiring decryption during computation time. Utilizing the technology will empower large-scale cross-institution and cross- stakeholder collaboration, allowing safe international collaborations. Clinical trial information: 0048-19-TLV. [Table: see text]

This paper presents the ‘hyper-sinh’, a variation of the m-arcsinh activation function suit-able for Deep Learning (DL)-based algorithms for supervised learning, including Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN), such as the Long Short-Term Memory (LSTM). hyper-sinh, developed in the open-source Python libraries TensorFlow and Keras, is thus described and validated as an accurate and reliable activation function for shallow and deep neural networks. Improvements in accuracy and reliability in image and text classification tasks on six (N=6) medium-to-large open-source benchmark datasets are discussed. Experimental results demonstrate that the overall competitive classification performance of the novel hyper-sinh function on shallow and deep neural networks yielded the highest performance. Furthermore, this activation is evaluated against other gold standard activation functions, demonstrating its overall competitive accuracy and reliability for both image and text classification tasks.

Secure computation, a methodology of computing on encrypted data, has become a key factor in machine learning. Homomorphic encryption (HE) enables computation on encrypted data without leaking any information to untrusted servers. In machine learning, the model selection method is a crucial algorithm that determines the performance and reduces the fitting problem. Despite the importance of finding the optimal model, none of the previous studies have considered model selection when performing data analysis through the HE scheme. The HE-based model selection we proposed finds the optimal complexity that best describes given data that is encrypted and whose distribution is unknown. Since this process requires a matrix calculation, we constructed the matrix multiplication and inverse of the matrix based on the bitwise operation. Based on these, we designed the model selection of the HE cross-validation approach and the HE Bayesian approach for homomorphic machine learning. Our focus was on evidence approximation for linear models to find goodness-of-fit that maximizes the evidence. We conducted an experiment on a dataset of age and Body Mass Index (BMI) from Kaggle to compare the capabilities and our model showed that encrypted data can regress homomorphically without decrypting it.

Abstract: Encryption is essential in preventing unauthorized access to sensitive data in light of the growing concerns about data security in cloud computing. Homomorphic encryption promises to enable secure calculations on encrypted data without the need for decryption, particularly for cloud-based operations. To evaluate the effectiveness and applicability of several homomorphic encryption algorithms for safe cloud computing, we compare and contrast them in this research paper. Partially homomorphic encryption (PHE), somewhat homomorphic encryption (SHE), and fully homomorphic encryption (FHE) are the three basic homomorphic encryption subtypes that we examine. The implications of this study can aid cloud service providers and organizations in selecting the most appropriate homomorphic encryption scheme based on their specific security requirements and performance considerations. The research contributes to the ongoing efforts to enhance data privacy in cloud computing environments, opening new possibilities for secure data processing in an increasingly connected digital world. The exploration of homomorphic encryption schemes in this study opens new avenues for research and development in the field of cryptographic techniques. As technology continues to evolve, so too must our approaches to safeguarding data. This research serves as a catalyst for further innovations in homomorphic encryption algorithms, enabling even more efficient and robust methods for secure data processing in cloud environments and beyond. The insights derived from this research paper not only empower cloud service providers and organizations to make informed decisions about selecting the most appropriate homomorphic encryption scheme but also contribute to the broader mission of fortifying data privacy and security in cloud computing.

Recent years have seen a significant amount of studies in the area of speech enhancement. This review looks at several speech improvement methods as well as Deep Neural Network (DNN) functions in speech enhancement. Speech transmissions are frequently distorted by ambient noise, background noise, and reverberations. There are processing methods, such as Short-time Fourier Transform, Short-time Autocorrelation, and Short-time Energy (STE), that can be used to enhance speech. To reduce speech noise, features such as the Mel-Frequency Cepstral Coefficients (MFCCs), Logarithmic Power Spectrum (LPS), and Gammatone Frequency Cepstral Coefficients (GFCCs) can be retrieved and input to a DNN. DNN is essential to speech improvement since it builds models using a lot of training data and evaluates the efficacy of the enhanced speech using certain performance metrics. Since the beginning of deep learning publications in 1993, a variety of speech enhancement methods have been examined in this study. This review provides a thorough examination of the several neural network topologies, training algorithms, activation functions, training targets, acoustic features, and databases that were employed for the job of speech enhancement and were gathered from various articles published between 1993 and 2022.

Many industrial and real-life problems exhibit highly nonlinear periodic behaviors and the conventional methods may fall short of finding their analytical or closed-form solutions. Such problems demand cutting-edge computational tools with increased functionality and reduced cost. Recently, deep neural networks have gained massive research interest due to their ability to handle large data and universality to learn complex functions. In this work, we put forward a methodology based on deep neural networks with responsive layers structure to deal nonlinear oscillations in microelectromechanical systems. We incorporated some oscillatory and non-oscillatory activation functions such as growing cosine unit known as GCU, Sine, Mish and Tanh in our designed network to have a comprehensive analysis on their performance for highly nonlinear problems. Integrating oscillatory activation functions with deep neural networks definitely outperform in predicting the periodic patterns of underlying systems. To support oscillatory actuation for nonlinear systems, we have proposed a novel oscillatory activation function called Amplifying Sine Unit denoted as ASU which is more efficient than GCU for complex vibratory systems such as microelectromechanical systems. Experimental results show that the designed network with our proposed activation function ASU is more reliable and robust to handle the challenges posed by oscillations

Privacy-preserving machine learning (PPML) demands secure inference on sensitive data in untrusted environments. Fully homomorphic encryption (FHE) enables computation directly on encrypted data, but it incurs prohibitive communication and computational overhead on edge devices. Hybrid homomorphic encryption (HHE) mitigates this by combining symmetric encryption with FHE, reducing client-side costs; however, existing implementations remain impractical for resource-constrained deployments. This paper proposes the first end-to-end hardware-accelerated HHE framework, which integrates a lightweight symmetric cipher optimized for FHE compatibility with a dedicated hardware accelerator. Beyond this integration, we introduce a microarchitectural optimization for throughput and energy efficiency. The proposed architecture is validated within a complete PPML pipeline, demonstrating significantly lower latency and power consumption than software implementations. Experimental results on a PYNQ-Z2 platform with MNIST validate our approach, achieving a 50× reduction in client-side encryption latency and a 2× throughput gain over existing FPGA-based HHE accelerators. These improvements enable practical and secure inference on edge devices. Our contributions establish a hardware-software co-design methodology for deploying scalable, secure machine learning in resource-constrained environments, validating the feasibility of low-power HHE for edge deployment.

Secure two-party computation (2PC) based on homomorphic encryption (HE) achieves formal data privacy protection and gets increasing adoption for private deep neural network (DNN) inference. As modern HE schemes usually operate on polynomials, existing works rely on manually-designed HE kernels for representative DNN operations. However, this is not only unscalable considering the diverse operator types, shapes, polynomial orders, etc, but also misses important optimization opportunities. In this paper, we introduce FlexHE, a flexible kernel generation framework to enable automatic generation and optimization of HE kernels for 2PC-based private inference. Given a high-level description of DNN operations, FlexHE can systematically define the HE kernel design space considering various optimization dimensions, including loop tiling, reordering, etc. We also analyze the communication and computation impact of different optimization dimensions for design space reduction. To search for the best kernel design, a two-level optimization problem is formulated and iteratively solved with an integer linear programming (ILP) formulation. With extensive experimental results, we not only demonstrate a better coverage of DNN operations including depth-wise Conv3D and dilated Conv3D, but also achieve more than 100×, 7.9×, and 4.2× latency reduction compared to prior-art HElayers, Cheetah, and Falcon, respectively.

Deep learning models, particularly those used for im-age classification, have become pervasive in today's technological landscape. However, the increasing demand for privacy, espe-cially when sharing data with external parties, poses significant challenges. Homomorphic encryption (HE) offers a compelling solution by enabling computations on encrypted data without compromising privacy. Despite its potential, the widespread adoption of HE remains limited due to its computational complexity. In this work, we propose an optimized approach to improve the efficiency of a fundamental building block in image classification models: the convolution operation. Our approach is based on the observation that multiple kernel evaluations in a convolution layer can be performed in parallel, provided that their receptive fields do not overlap. Using this property, we reorganize convolution computations into independent groups, significantly reducing redundant operations. This grouping reduces the number of multiplications by up to 75% compared to standard encrypted convolution techniques. Additionally, the approach eliminates the need for costly preprocessing, enabling direct processing of encrypted inputs. To further accelerate homomorphic operations, we leverage a GPU-accelerated HE library, enabling faster op-erations. These contributions make encrypted convolution signifi-cantly more practical for real-world deep learning applications.

Artificial neural networks (ANN) have been used in many applications such like handwriting recognition and speech recognition. It is well-known that learning rate is a crucial value in the training procedure for artificial neural networks. It is shown that the initial value of learning rate can confoundedly affect the final result and this value is always set manually in practice. A new parameter called beta stabilizer has been introduced to reduce the sensitivity of the initial learning rate. But this method has only been proposed for deep neural network (DNN) with sigmoid activation function. In this paper we extended beta stabilizer to long short-term memory (LSTM) and investigated the effects of beta stabilizer parameters on different models, including LSTM and DNN with relu activation function. It is concluded that beta stabilizer parameters can reduce the sensitivity of learning rate with almost the same performance on DNN with relu activation function and LSTM. However, it is shown that the effects of beta stabilizer on DNN with relu activation function and LSTM are fewer than the effects on DNN with sigmoid activation function.