A Novel Differentially Private Online Learning Algorithm for Group Lasso in Big Data

Photo by Josh Riemer on Unsplash
 Introduction
 With the rapid advancements in neurotechnological machinery and improved analytical insights from machine learning in neuroscience, the availability of big brain data has increased tremendously. Neurological health research is done using digitized brain data.[1] There must be adequate data governance to secure the privacy of subjects participating in brain research and treatments. If not properly regulated, the research methods could lead to significant breaches of the subject’s autonomy and privacy. This paper will address the necessity for neuroprotection laws, which effectively govern the use of big brain data to ensure respect for patient privacy and autonomy.
 Background
 Artificial intelligence and machine learning can be integrated with neuroscience big brain data to drive research studies. This integrative technology allows patterns of electrical activity in neurons to be studied in detail.[2]Specifically, it uses a robotic system which can reason, plan, and exhibit biologically intelligent behavior. Machine learning is a method of computer programming where the code can adapt its behavior based on big brain data.[3] The big brain data is the collection of large amounts of information for the purpose of deciphering patterns through computer analysis using machine learning.[4] The information that these technologies provide is extensive enough to allow a researcher to read a patient’s mind. AI and machine learning technologies work by finding the underlying structure of brain data, which is then described by patterns known as latent factors, eventually resulting in an understanding of the brain’s temporal dynamics.[5]
 Through these technologies, researchers are able to decipher how the human brain computes its performances and thoughts. However, due to the extensive and complex nature of the data processed through AI and machine learning, researchers may gain access to personal information a patient may not wish to reveal. From a bioethical lens, tensions arise in the realm of patient autonomy. Patients are not able to control the transmission of data from their brains that is analyzed by researchers. Governing brain data through laws may enhance the extent of patient privacy in the case where brain data is being used through AI technologies.[6] A responsible approach to governing brain data would require a sophisticated legal structure.
 Analysis
 Impact on Patient Autonomy and Privacy 
 In research pertaining to big brain data, the consent forms do not fully cover the vast amounts of information that is collected. According to research, personal data has become the most sought out commodity to provide content to corporations and the web-based service industry. Unfortunately, data leaks that release private information frequently occur.[7] The storage of an individual’s data on technologies accessible on the internet during research studies makes it vulnerable to leaks, jeopardizing an individual’s privacy. These data leaks may cause the patient to be identified easily, as the degree of information provided by AI technologies are personalized and may be decoded through brain fingerprinting methods.[8]
 There has been an extensive growth in the development and use of AI. It is efficient in providing information to radiologists who diagnose various diseases including brain cancer and psychiatric disease, and AI assists in the delivery of telemedicine.[9] However, the ethical pitfall of reduced patient autonomy must be addressed by analyzing current AI technologies and creating more options for patient preference in how the data may be used. For instance, facial recognition technology[10] commonly used in health care produces more information than listed in common consent forms, threatening to undermine informed consent. Facial recognition software collects extensive data and may disclose more information than a person would prefer to provide despite being a useful tool for diagnosing medical and genetic conditions.[11] In addition, people may not be aware that their images are being used to generate more clinical data for other purposes. It is difficult to guarantee the data is anonymized. Consent requirements must include informing people about the complexity of the potential uses of the data; software developers should maximize patient privacy.[12] Furthermore, there is a “human element” in the use of AI technologies as medical providers control the use and the extent to which data is captured or accessed through the AI technologies.[13] People must understand the scope of the technology and have clear communication with the physician or health care provider about how the medical information will be used. 
 Existing Laws for Brain Data Governance 
 A strict system of defined legal responsibilities of medical providers will ensure a higher degree of patient privacy and autonomy when AI technologies and data from machine learning are used. Governing specific algorithmic data is crucial in safeguarding a patient’s privacy and developing a gold standard treatment protocol following the procurement of the information.[14] Certain AI technologies provide more data than others, and legal boundaries should be established to ensure strong performance, quality control, and scope for patient privacy and autonomy. For instance, currently AI technologies are being used in the realm of intensive neurological care. However, there is a significant level of patient uncertainty about how much control patients have over the data’s uses.[15] Calibrated legal and ethical standards will allow important brain data to be securely governed and monitored.
 Once brain signals are recorded and processed from one individual, the data may be merged with other data in Brain Computer Interface Technology (BCI).[16] To ensure a right and ability to retrieve personal data or pull it from the collection, specific regulations for varying types of data are needed.[17] The importance of consent and patient privacy must be considered through giving patients a transparent view of how brain data is governed.[18] The legal system must address discriminatory issues and risks to patients whose data is used in studies. Laws like the General Data Protection Regulation (GDPR) and the California Consumer Privacy Protection Act (CCPA) can serve as effective models to protect aggregated data. These laws govern consumer information and ensure the compliance when personal data is collected.[19] California voters recently approved expansion of the CCPA to health data. The Washington Privacy Act, which would have provided rights to access, change, and withdraw personal data, failed to pass. Other states should improve privacy as well,[20] although a federal bill would be preferable. Scientists at the Heidelberg Academy of Sciences argue for data security to be governed in a manner that balances patient privacy and autonomy with the commercial interests of researchers.[21] The balance could be achieved through privacy protections like those in the Washington Privacy Act. Although the Health Insurance Portability and Accountability Act (HIPAA) provides an overall framework to deter the likelihood of dangers to patient protection and privacy, more thorough laws are warranted to combat pervasive data transfer and analysis that technology has brought to the health care industry.[22] Breaches of patient privacy under current HIPAA regulations include releasing patient information to a reporter without their consent and sending HIV data to a patient’s employer without consent.[23] HIPAA does not cover information being shared with outside contractors who do not have an agreement with technology companies to keep patient data confidential. HIPAA regulations also do not always address blatant breaches on patient data confidentiality.[24] Patients must be provided with methods to monitor the data being analyzed to be able to view the extent of private information being generated via AI technologies. In health research, the medical purposes of better diagnosis, earlier detection of diseases, or prevention are ethical justifications for the use of the data if it was collected with permission, the person understood and approved the uses of the data, and the data was deidentified.
 A standard governance framework is required in providing the fairest system of care to patients who allow their brain data to be examined. Informed consent in the neuroscience field could reaffirm the privacy and autonomy of patients by ensuring that they understand the type of information collected. Laws also could protect data after a patient’s death. Malpractice in the scope of brain data could give people a cause of action critical in safeguarding patient’s rights. Data breach lawsuits will become common but generally do not cover deidentified data that becomes part of big data collection. A more synchronized approach to the collection and consent process will encourage an understanding of how big data is used to diagnose and treat patients. Some altruistic people may even be more likely to consent if they know the largescale data collection is helpful to treat and diagnose people. Others should have the ability to opt out of sharing neurological data, especially when there is not certainty surrounding deidentification.[25]
 Conclusion
 Artificial intelligence and machine learning technologies have the potential to aid in the diagnosis and treatment of people globally by extracting and aggregating brain data specific to individuals. However, the secure use of the data is necessary to build trust between care providers and patients, as well as in balancing the bioethical principles of beneficence and patient autonomy. We must ensure the highest quality of care to patients, while protecting their privacy, informed consent, and clinical trust. More sophis

This chapter presents a survey of the most important security and privacy issues related to large-scale data sharing and mining in big data with focus on differential privacy as a promising approach for achieving privacy especially in statistical databases often used in healthcare. A case study is presented utilizing differential privacy in healthcare domain, the chapter analyzes and compares the major differentially private data release strategies and noise mechanisms such as the Laplace and the exponential mechanisms. The background section discusses several security and privacy approaches in big data including authentication and encryption protocols, and privacy preserving techniques such as k-anonymity. Next, the chapter introduces the differential privacy concepts used in the interactive and non-interactive data sharing models and the various noise mechanisms used. An instrumental case study is then presented to examine the effect of applying differential privacy in analytics. The chapter then explores the future trends and finally, provides a conclusion.

With the advent of the volume, velocity and variety of data in today’s day and age, big data has become the norm in many places. Data at rest, the same data that is fed to the analyst of a company is at most times, not encrypted and depends on the fact that a person can be trusted by the company. Our contribution is implementing differential privacy in a NoSQL database while combating the few drawbacks in differential privacy with techniques such as cluffering (clustering and shuffling) and k-Anonymization. We implement an end to end implementation in a NoSQL database, MongoDB to promote privacy by perturbation using differential privacy which has been shown to be one of the best ways to do so. A generic system gets formed where local differential privacy is enforced before sending the sensitive database for analysis and enforcing differential privacy concepts on aggregate queries ensuring that no particular individual gets exposed in the process. We further provide the basis for implementing other aggregates apart from the ones we implemented and for complex queries that are not yet supported while the core concepts remain the same.

Differential Privacy and Qualitative Privacy Analysis for Nonlinear Dynamical Systems

Big data is a term used for very large data sets that have more varied and complex structure. These characteristics usually correlate with additional difficulties in storing, analyzing and applying further procedures or extracting results. Big data analytics is the term used to describe the process of researching massive amounts of complex data in order to reveal hidden patterns or identify secret correlations. However, there is an obvious contradiction between the security and privacy of big data and the widespread use of big data. This paper focuses on privacy and security concerns in big data, differentiates between privacy and security and privacy requirements in big data. This paper covers uses of privacy by taking existing methods such as HybrEx, k-anonymity, T-closeness and L-diversity and its implementation in business. There have been a number of privacy-preserving mechanisms developed for privacy protection at different stages (for example, data generation, data storage, and data processing) of a big data life cycle. The goal of this paper is to provide a major review of the privacy preservation mechanisms in big data and present the challenges for existing mechanisms. This paper also presents recent techniques of privacy preserving in big data like hiding a needle in a haystack, identity based anonymization, differential privacy, privacy-preserving big data publishing and fast anonymization of big data streams. This paper refer privacy and security aspects healthcare in big data. Comparative study between various recent techniques of big data privacy is also done as well.

Online learning is a machine learning paradigm that is useful when data is not available all at once. In this paper we focus in real time applications for which data is being collected as the algorithm executes. The forgetron Algorithm [1] is an online learning algorithm that works under a limited memory constraint while guaranteeing a bound on the number of total mistakes. We have proposed a specific architecture for the forgetron algorithm using hardware/software based design in order to improve computation time in the training process. Experiments on real world data show the advantages of this implementation compared to an exclusive software implementation using a Xilinx Virtex II Pro FPGA with an embedded Power PC. These experiments validate the performance of the proposed architecture.

The rise of the ``big data'' era has created a pressing demand for educating many data scientists and engineers quickly at low cost. It is essential they learn by working on assignments that involve real world data sets to develop the skills needed to be successful in the workplace. However, enabling instructors to flexibly deliver all kinds of data science assignments using real world data sets to large numbers of learners (both on-campus and off-campus) at low cost is a significant open challenge. To address this emerging challenge generally, we develop and deploy a novel Cloud-based Lab for Data Science (CLaDS) to enable many learners around the world to work on real-world data science problems without having to move or otherwise distribute prohibitively large data sets. Leveraging version control and continuous integration, CLaDS provides a general infrastructure to enable any instructor to conveniently deliver any hands-on data science assignment that uses large real world data sets to as many learners as our cloud-computing infrastructure allows at very low cost. In this paper, we present the design and implementation of CLaDS and discuss our experience with using CLaDS to deploy seven major text data assignments for students in both an on-campus course and an online course to work on for learning about text data retrieval and mining techniques; this shows that CLaDS is a very promising novel general infrastructure for efficiently delivering a wide range of hands-on data science assignments to a large number of learners at very low cost.

One of the core issues across computer and computational science today is adapting to, managing, and learning from the influx of Big Data. In the commercial space, this problem has led to a huge investment in new technologies and capabilities that are well adapted to dealing with the sorts of human-generated logs, videos, texts, and other large-data artifacts that are processed and resulted in an explosion of useful platforms and languages (Hadoop, Spark, Pandas, etc.). However, translating this work from the enterprise space to the computational science and HPC community has proven somewhat difficult, in part because of some of the fundamental differences in type and scale of data and timescales surrounding its generation and use. We describe a forward-looking research and development plan which centers around the concept of making Input/Output (I/O) intelligent for users in the scientific community, whether they are accessing scalable storage or performing in situ workflow tasks. Much of our work is based on our experience with the Adaptable I/O System (ADIOS 1.X), and our next generation version of the software ADIOS 2.X [1].

Big Data Focus on Physicians: Start-up Profiles Physicians’ Pharma Friendliness

Using the machine learning technology to explore the potential value of Big Data brings us into a smarter world, and the way data is mined through data sharing patterns also threatens the privacy of personal data. Differential privacy is a prevalent mechanism to effectively protect the personal data privacy due to the strict and the provable privacy definition, although there are several achievements have reached by combining the differential privacy and traditional machine learning algorithms in a stand-alone mode, little to talk about the distributed environment. To fill this gap, this paper proposes a method to embed the differential privacy mechanism into distributed platform, respectively implements the DPLloyd, GUPT k-means and GUPT logistic regression on the platform of Spark. The evaluation demonstrates that the approach barely interferes the effect of distributed machine learning algorithms and thus achieves the goal of differential privacy.

FAST ONLINE TRAINING OF L1 SUPPORT VECTOR MACHINES By Gabriella Melki A thesis submitted in partial fulfillment of the requirements for the degree of Master of Science at Virginia Commonwealth University. Virginia Commonwealth University, 2016. Directors: Vojislav Kecman, Alberto Cano Department of Computer Science This thesis proposes a novel experimental environment (non-linear stochastic gradient descent, NL-SGD), as well as a novel online learning algorithm (OL SVM), for solving a classic nonlinear Soft Margin L1 Support Vector Machine (SVM) problem using a Stochastic Gradient Descent (SGD) algorithm. The NL-SGD implementation has a unique method of random sampling and alpha calculations. The developed code produces a competitive accuracy and speed in comparison with the solutions of the Direct L2 SVM obtained by software for Minimal Norm SVM (MN-SVM) and Non-Negative Iterative Single Data Algorithm (NN-ISDA). The latter two algorithms have shown excellent performances on large datasets; which is why we chose to compare NL-SGD and OL SVM to them. All experiments have been done under strict double (nested) cross-validation, and the results are reported in terms of accuracy and CPU times. OL SVM has been implemented within MATLAB and is compared to the classic Sequential Minimal Optimization (SMO) algorithm implemented within MATLAB’s solver, fitcsvm. The experiments with OL SVM have been done using k-fold cross-validation and the results reported in % error and % speedup of CPU Time. viii

Workplace wellness programs come in a myriad of forms, each with the goal of improving employee health and productivity while reducing healthcare costs. In the age of big data, wearable devices are ubiquitously incorporated into workplace wellness programs. Wearable devices in wellness programs can be beneficial for employers, employees, and health insurers alike. Nevertheless, there is an increasingly complex risk landscape associated with wearable devices in wellness programs, raising profound legal and ethical concerns related to privacy, security, information abuse, and employee autonomy. This paper will discuss the benefits and challenges of wearable devices in workplace wellness programs. Part I will introduce the benefits of workplace wellness programs. Part II will discuss the incorporation of wearable technologies in workplace wellness programs. Part III will analyze the legal and ethical challenges associated with the use of wearable technologies in wellness programs. Finally, Part IV will propose soft law, or best practices, as the most efficacious governance mechanism for employers and employees to secure benefits and balance concerns associated with the use of wearable devices in workplace wellness programs.

A private data federation enables clients to query the union of data from multiple data providers without revealing any extra private information to the client or any other data providers. Unfortunately, this strong end-to-end privacy guarantee requires cryptographic protocols that incur a significant performance overhead as high as 1,000 x compared to executing the same query in the clear. As a result, private data federations are impractical for common database workloads. This gap reveals the following key challenge in a private data federation: offering significantly fast and accurate query answers without compromising strong end-to-end privacy. To address this challenge, we propose SAQE, the Secure Approximate Query Evaluator, a private data federation system that scales to very large datasets by combining three techniques --- differential privacy, secure computation, and approximate query processing --- in a novel and principled way. First, SAQE adds novel secure sampling algorithms into the federation's query processing pipeline to speed up query workloads and to minimize the noise the system must inject into the query results to protect the privacy of the data. Second, we introduce a query planner that jointly optimizes the noise introduced by differential privacy with the sampling rates and resulting error bounds owing to approximate query processing. Our research shows that these three techniques are synergistic: sampling within certain accuracy bounds improves both query privacy and performance, meaning that SAQE executes over less data than existing techniques without sacrificing efficiency, privacy, or accuracy. Using our optimizer, we leverage this counter-intuitive result to identify an inflection point that maximizes all three criteria prior query evaluation. Experimentally, we show that this result enables SAQE to trade-off among these three criteria to scale its query processing to very large datasets with accuracy bounds dependent only on sample size, and not the raw data size.

In response to the privacy leakage risks inherent in the big data processing of power user personas, propose a collaborative optimization‐based joint privacy protection mechanism for Differential Privacy (DP) and Secret Sharing (SS). This method reveals the inherent correlation between DP noise parameters and SS sharding strategy and designs an adaptive noise sharding mapping mechanism to achieve joint optimization of privacy protection and communication overhead. Based on this mechanism, a specific privacy protection scheme for power user personas is designed. Experimental results demonstrate that the DP + SS privacy protection method designed in this study achieves an accuracy rate of 93.14% on the MNIST dataset and maintains an accuracy rate of 92.06% across multiple datasets, indicating that the scheme effectively quantifies the level of privacy protection. The privacy protection level of the scheme is quantitatively defined through differential privacy budget ε ∈ [0.11,0.15], secret sharing ( k = 3, n = 5) threshold mechanism, and privacy leakage probability δ = 10 −5 . The privacy budget is dynamically allocated through the correlation between differential privacy noise variance and the minimum reconstruction threshold of secret sharing, ensuring that privacy risks are controllable and quantifiable. At the same time, the average training time for servers is about 186.4 s, which refers to the total end‐to‐end training time in a four‐server cluster environment. Compared with a single server encryption scheme, the efficiency is improved by 48.6%. The communication overhead has increased to 10.6GB, which is the total chip transmission overhead per 10,000 users. However, the calculation stability can be maintained through load balancing, indicating that the scheme has certain feasibility. The aforementioned research results not only provide a viable applied cryptographic technology solution for privacy protection in power big data but also offer new insights into balancing privacy protection strength and data availability. More importantly, the joint privacy protection mechanism proposed by the research institute can serve as a fundamental enabling technology for the infrastructure of a secure smart grid, providing quantifiable and trustworthy privacy and security guarantees for advanced data‐driven applications such as precise load prediction, demand‐side response, and collaborative optimization of distributed energy, thereby supporting the reliable and intelligent development of the smart grid.

The macro nature of social networks has always been a hot topic of scientific research, which is of great help in studying the characteristics of human social behavior. The spectrum is closely related to the nature of the network and determines certain information, including community division, loop, diameter, degree and so on. To ensure the security of data, the differential privacy security framework is used to propose three algorithms for privacy protection of edge weights in spectrum query. Firstly, the basic concepts of edge weight neighbor graph and edge weight difference privacy are given. For the single singular value query, the L1 global sensitivity of the function is proved, and an algorithm satisfying ε-differential privacy is designed by using Laplace mechanism. It can guarantee the privacy of edge weights when we query someone singular value. However, it cannot provide reasonable privacy protection for multiple singular value queries. So the L2 global sensitivity of the query for multiple singular values is further proved, and the Gaussian mechanism is used to design an algorithm that satisfies (ε, δ)-differential privacy. Finally, to combine with multi-singular values query strategy and spectral decomposition, a novel graph data publishing algorithm which can guarantee edge weight (ε, δ)-differential privacy is proposed. To verify the availability of these methods, experimental tests have been carried out in both model networks and actual networks, which shows that the algorithm can better guarantee the availability of data.