Research issues in Privacy-preserving Record Linkage

ABSTRACT
 ObjectivesWhile record linkage has become a strategic research priority within Australia and internationally, legal and administrative issues prevent data linkage in some situations due to privacy concerns. Even current best practices in record linkage carry some privacy risk as they require the release of personally identifying information to trusted third parties. Application of record linkage systems that do not require the release of personal information can overcome legal and privacy issues surrounding data integration. Current conceptual and experimental privacy-preserving record linkage (PPRL) models show promise in addressing data integration challenges but do not yet address all of the requirements for real-world operations. This paper aims to identify and address some of the challenges of operationalising PPRL frameworks.
 ApproachTraditional linkage processes involve comparing personally identifying information (name, address, date of birth) on pairs of records to determine whether the records belong to the same person. Designing appropriate linkage strategies is an important part of the process. These are typically based on the analysis of data attributes (metadata) such as data completeness, consistency, constancy and field discriminating power. Under a PPRL model, however, these factors cannot be discerned from the encrypted data, so an alternative approach is required. This paper explores methods for data profiling, blocking, weight/threshold estimation and error detection within a PPRL framework.
 ResultsProbabilistic record linkage typically involves the estimation of weights and thresholds to optimise the linkage and ensure highly accurate results. The paper outlines the metadata requirements and automated methods necessary to collect data without compromising privacy. We present work undertaken to develop parameter estimation methods which can help optimise a linkage strategy without the release of personally identifiable information. These are required in all parts of the privacy preserving record linkage process (pre-processing, standardising activities, linkage, grouping and extracting).
 ConclusionsPPRL techniques that operate on encrypted data have the potential for large-scale record linkage, performing both accurately and efficiently under experimental conditions. Our research has advanced the current state of PPRL with a framework for secure record linkage that can be implemented to improve and expand linkage service delivery while protecting an individual’s privacy. However, more research is required to supplement this technique with additional elements to ensure the end-to-end method is practical and can be incorporated into real-world models.

ABSTRACTObjectiveIn most European settings, record linkage across different institutions has to be based on personal identifiers such as names, birthday or place of birth. To protect the privacy of research subjects, the identifiers have to be encrypted. In practice, these identifiers show error rates up to 20% per identifier, therefore linking on encrypted identifiers usually implies the loss of large subsets of the databases. In many applications, this loss of cases is related to variables of interest for the subject matter of the study. Therefore, this kind of record-linkage will generate biased estimates.
 These problems gave rise to techniques of Privacy Preserving Record Linkage (PPRL). Many different PPRL techniques have been suggested within the last 10 years, very few of them are suitable for practical applications with large database containing millions of records as they are typical for administrative or medical databases. One proven technique for PPRL for large scale applications is PPRL based on Bloom filters.MethodUsing appropriate parameter settings, Bloom filter approaches show linkage results comparable to linkage based on unencrypted identifiers. Furthermore, this approach has been used in real-world settings with data sets containing up to 100 Million records. By the application of suitable blocking strategies, linking can be done in reasonable time.ResultHowever, Bloom filters have been subject of cryptographic attacks. Previous research has shown that the straight application of Bloom filters has a nonzero re-identification risk. We will present new results on recently developed techniques to defy all known attacks on PPRL Bloom filters. These computationally simple algorithms modify the identifiers by different cryptographic diffusion techniques. The presentation will demonstrate these new algorithms and show their performance concerning precision, recall and re-identification risk on large databases.

IntroductionResearch data combined with administrative data provides a robust resource capable of answering unique research questions. However, in cases where personal health data are encrypted, due to ethics requirements or institutional restrictions, traditional methods of deterministic and probabilistic record linkages are not feasible. Instead, privacy-preserving record linkages must be used to protect patients’ personal data during data linkage.ObjectivesTo determine the feasibility and validity of a deterministic privacy preserving data linkage protocol using homomorphically encrypted data.MethodsFeasibility was measured by the number of records that successfully matched via direct identifiers. Validity was measured by the number of records that matched with multiple indirect identifiers. The threshold for feasibility and validity were both set at 95%. The datasets shared a single, direct identifier (health card number) and multiple indirect identifiers (sex and date of birth). Direct identifiers were encrypted in both datasets and then transferred to a third-party server capable of linking the encrypted identifiers without decrypting individual records. Once linked, the study team used indirect identifiers to verify the accuracy of the linkage in the final dataset.ResultsWith a combination of manual and automated data transfer in a sample of 8,128 individuals, the privacy-preserving data linkage took 36 days to match to a population sample of over 3.2 million records. 99.9% of the records were successfully matched with direct identifiers, and 99.8% successfully matched with multiple indirect identifiers. We deemed the linkage both feasible and valid.ConclusionsAs combining administrative and research data becomes increasingly common, it is imperative to understand options for linking data when direct linkage is not feasible. The current linkage process ensured the privacy and security of patient data and improved data quality. While the initial implementations required significant computational and human resources, increased automation keeps the requirements within feasible bounds.

Big Data projects often require data from several sources to be integrated before they can be used for analysis. Once data have been integrated, they allow more detailed analysis that would otherwise not be possible. Accordingly, recent years have seen an increasing interest in techniques that facilitate the integration of data from diverse sources. Whenever data about individuals, or otherwise sensitive data, are to be integrated across organizations, privacy and confidentiality have to be considered. Domains where privacy preservation during data integration is of importance include business collaborations, health research, national censuses, the social sciences, crime and fraud detection, and homeland security. Increasingly, applications in these domains require data from diverse sources (both internal and external to an organization) to be integrated.Consequently, in the past decade, various techniques have been developed that aim to facilitate data integration without revealing any private or confidential information about the databases and records that are integrated. These techniques either provably prevent leakage of any private information, or they provide some empirical numerical measure of the risk of disclosure of private information.In the first part of this presentation we provide a background on data integration, and illustrate the importance of preserving privacy during data integration with several application scenarios. We then given an overview of the main concepts and techniques that have been developed to facilitate data integration in such ways that no private or confidential information is being revealed. We focus on privacy-preserving record linkage (PPRL), where so far most research has been conducted. We describe the basic protocols used in PPRL, and several key technologies employed in these protocols. Finally, we discuss the challenges privacy poses to data integration in the era of Big Data, and we discuss directions and opportunities in this research area.

Record linkage represents the process of identifying records that refer to the same real-world entity across multiple sources. In the recent past, the large explosion of data from organizations and individuals has created critical challenges in record linkage process, leading the scientific community to develop a rich series of techniques. Among these recent developments, the design of record linkage solutions for medical data is a cornerstone for health-care systems, as personal identifying information, such as name and date of birth, is often used for linkage. In this setting, record linkage solutions are expected to preserve individual patients’ privacy in addition to be effective and efficient. In this chapter, we provide a broad review of the recent works in privacy preserving record linkage (PPRL). From a privacy-centric perspective, we summarize a comprehensive framework of the PPRL process and describe the privacy assurance techniques adopted for each step in the framework. With the applications in biomedical domain in mind, we identify several challenges for PPRL and discuss future research directions.

BackgroundData on newborns is regularly linked for epidemiological research. However, hospital data often suffers from incomplete data. We report on a linkage of two population-covering administrative health databases containing neonatal and perinatal data without unique personal identifiers and with incomplete information in standard patient identifiers.
 GoalTo study the effects of a policy-induced change from linking a national database without standard patient identifiers to a privacy-preserving Record Linkage method, we compare the linkage system in use to clear-text and privacy-preserving Record Linkage techniques. We expected large proportions of missing identifiers since they are not needed for clinical practice. Therefore, we expected missing links caused by missing identifiers. To study the impact of these missing identifiers on these successful links, we compared several linkage methods. Furthermore, we study the variations of linkage success between hospitals.
 MethodsPerinatal and neonatal data from population-covering real-world administrative databases was linked using several variants of state of the art methods, including Privacy-preserving Record Linkage (PPRL) techniques such as multiple match keys and Bloom filter methods.
 Results We report on the variation of linkage results between the hospitals and give possible explanations for the differences.
 The resulting linkage success is reported for each method. The impact of incomplete data on linkage success for each method is documented.
 Finally, we report on the relative performance of the modified techniques compared to standard linkage procedures used in practice.
 ConclusionImplementing a record linkage system based on identifiers not required for clinical practice caused a large number of missing identifiers. Since this information is essential for successful clear-text and private linkage methods, emphasizing the need for documenting patient identifiers, especially in cases where auxiliary information (such as stable addresses, date of birth or health insurance numbers) are missing, is of central importance for implementing a privacy-preserving Record Linkage system.

IntroductionBusinesses worldwide are increasingly adopting the storage, compute and analytical services provided by cloud computing. Yet, few operational linkage units are keeping pace with this world of technological change - most use legacy systems approaching their limits with the rapidly increasing size and range of datasets now required for linkage.
 Objectives and ApproachTo meet the demands of linkage for the near future, it is important that new solutions for linkage consider the services provided by public cloud infrastructure for compute, storage and analytics. We examined Platform as a Service (PaaS) offerings for use in the development of a cost-effective cloud model for scalable, privacy-preserving record linkage (PPRL). PPRL techniques were adapted to maximise the quality of linkage and to automate as much of the process as possible. Finally, a prototype was created to demonstrate the capabilities and potential of the model.
 ResultsWe present our cloud model for PPRL, a platform for record linkage that provides rapid scaling of resources to meet demand, and the results of how our prototype performed on massive datasets.
 Conclusion/ImplicationsThe application of record linkage using relatively inexpensive cloud infrastructure represents a significant step towards providing an efficient and scalable record linkage service to researchers and government. Larger datasets can be linked efficiently, including national or cross-jurisdictional datasets, with little investment in private infrastructure, and improved turnaround times for researchers.

Record linkage is an emerging research area which is required by various real-world applications to identify which records in different data sources refer to the same real-world entities. Often privacy concerns and restrictions prevent the use of traditional record linkage applications across different organizations. Linking records in situations where no private or confidential information can be revealed is known as privacy-preserving record linkage (PPRL). As with traditional record linkage applications, scalability is a main challenge in PPRL. This challenge is generally addressed by employing a blocking technique that aims to reduce the number of candidate record pairs by removing record pairs that likely refer to non-matches without comparing them in detail. This paper presents an efficient private blocking technique based on a sorted neighborhood approach that combines k-anonymous clustering and the use of public reference values. An empirical study conducted on real-world databases shows that this approach is scalable to large databases, and that it can provide effective blocking while preserving k-anonymous characteristics. The proposed approach can be up-to two orders of magnitude faster than two state-of-the-art private blocking techniques, k-nearest neighbor clustering and Hamming based locality sensitive hashing.

ObjectiveTo introduce Linkja, an open-source, certified, Privacy-Preserving Record Linkage (PPRL) tool with the primary objective of facilitating secure and efficient record linkage across diverse datasets, while preserving individual privacy, providing a comprehensive and privacy-centric approach to program assessments. ApproachLinkja has been used to evaluate Chicago’s Flexible Housing Pool program by securely linking various datasets, including Homeless Management Information Systems (HMIS), Electronic Health Records, Jail records, Medical Examiner reports, and Public Health data. Data contributors used separate honest brokers for record linkage, and generation of salt and cryptography keys. ResultsLinkja enabled this multi-sector linkage by facilitating legal and data sharing agreements, avoided licensing fees and procurement processes for all parties, and provided a de-identified for analysis. The insights gleaned from the analysis motivated the FHP program to plan a follow-up evaluation that will add emergency serviced records from a separate City of Chicago department. The successful linkage provided a nuanced understanding of the association between stable housing and reduced healthcare utilization. ConclusionsLinkja's cryptographic certification ensures the highest standards of security during data linkage, safeguarding sensitive information. The matching rules, validated on hard-to-match populations, prove essential in maintaining accuracy and reliability in the linkage process. ImplicationsThe open-source nature of Linkja promotes transparency and collaboration in the development and enhancement of privacy-preserving record linkage techniques. This tool has broad implications for sectors dealing with sensitive data, fostering responsible data-sharing practices, and enabling secure collaborative research initiatives without compromising individual privacy.

Nearly 90% of today’s data have been produced only in the last two years! These data come from a multitude of human activities, including social networking sites, mobile phone applications, electronic medical records systems, e-commerce sites, etc. Integrating and analyzing this wealth and volume of data offers remarkable opportunities in sectors that are of high interest to businesses, governments, and academia. Given that the majority of the data are proprietary and may contain personal or business sensitive information, Privacy-Preserving Record Linkage (PPRL) techniques are essential to perform data integration. In this paper, we review existing work in PPRL, focusing on the computational aspect of the proposed algorithms, which is crucial when dealing with Big data. We propose an analysis tool for the computational aspects of PPRL, and characterize existing PPRL techniques along five dimensions. Based on our analysis, we identify research gaps in current literature and promising directions for future work.

The process of merging data of different data sources is referred to as record linkage. A medical environment with increased preconditions on privacy protection demands the transformation of clear-text attributes like first name or date of birth into one-way encrypted pseudonyms. When performing an automated or privacy preserving record linkage there might be the need of a binary classification deciding whether two records should be classified as the same entity. The classification is the final of the four main phases of the record linkage process: Preprocessing, indexing, matching and classification. The choice of binary classification techniques in dependence of project specifications in particular data quality has not extensively been studied yet. The aim of this work is the introduction and evaluation of an automatable semi-supervised binary classification system applied within the field of record linkage capable of competing or even surpassing advanced automated techniques of the domain of unsupervised classification. This work describes the rationale leading to the model and the final implementation of an automatable semi-supervised binary classification system and the comparison of its classification performance to an advanced active learning approach out of the domain of unsupervised learning. The performance of both systems has been measured on a broad variety of artificial test sets (n = 400), based on real patient data, with distinct and unique characteristics. While the classification performance for both methods measured as F-measure was relatively close on test sets with maximum defined data quality, 0.996 for semi-supervised classification, 0.993 for unsupervised classification, it incrementally diverged for test sets of worse data quality dropping to 0.964 for semi-supervised classification and 0.803 for unsupervised classification. Aside from supplying a viable model for semi-supervised classification for automated probabilistic record linkage, the tests conducted on a large amount of test sets suggest that semi-supervised techniques might generally be capable of outperforming unsupervised techniques especially on data with lower levels of data quality.

Record linkage aims to identify records referring to the same individual across different datasets, but often involves sensitive personal identifiers. Privacy-Preserving Record Linkage (PPRL) techniques, particularly those based on Bloom filter encoding, enable secure linkage while protecting privacy. However, existing methods suffer from information loss and lack rigorous privacy guarantees. To address these limitations, we propose a novel (Hash)-A method that retains position and frequency information of q-grams, enhancing matching accuracy. We also introduce the Utility-optimized Bloom Filter (UBF), a privacy-preserving mechanism grounded in User-Level Local Differential Privacy (ULDP), which offers formal and quantifiable privacy guarantees. These components are integrated into a complete three-party PPRL protocol that supports missing value handling. Experimental results on real-world datasets show that our method achieves improved linkage accuracy and robust privacy protection against inference attacks, demonstrating a practical balance between utility and privacy.

Privacy-preserving record linkage on large real world datasets

Privacy Preserving Record Linkage (PPRL) is the scientific field that explores methods of linking datasets in order to identify common entities efficiently and accurately by simultaneously preserving the privacy of the underlying data. In this paper we present a distributed Locality Sensitive Hashing-based framework for linking huge collections of records, by grouping similar records efficiently and by distributing computations among underutilized commodity hardware resources uniformly, without imposing an extra overhead on the existing infrastructure, thus promoting scalability. We also propose two methods of assessing computational cost, aiming to distribute workload evenly among compute nodes.

Privacy-preserving record linkage (PPRL) can improve the utility of clinical data for research and discovery by merging disparate data sources to provide a more complete record of a patient. We assess the performance of record linkage across two data sets consisting of cancer registry data and electronic health records (EHRs). Using two data sources-EHR extracts and records from the Missouri Cancer Registry (MCR)-we performed various validation exercises, including an audit by the MCR and a comparative evaluation using multiple matching algorithms. Results indicated substantial matching performance, with high recall and precision rates. We conclude that the refined data linkage process significantly supports enhanced research capabilities without compromising patient privacy, thereby enabling reliable data integration for clinical research in oncology.