A Framework for Improving Trust in Dynamic Service-Oriented Systems

Theme: Data and Linkage QualityObjectives:
 
 To define health data quality from clinical, data science, and health system perspectives
 To describe some of the international best practices related to quality and how they are being applied to Canada’s administrative health data.
 To compare methods for health data quality assessment and improvement in Canada (automated logical checks, chart quality indicators, reabstraction studies, coding manager perspectives)
 To highlight how data linkage can be used to provide new insights into the quality of original data sources
 To highlight current international initiatives for improving coded data quality including results from current ICD-11 field trials
 
 Dr. Keith Denny: Director of Clinical Data Standards and Quality, Canadian Insititute for Health Information (CIHI), Adjunct Research Professor, Carleton University, Ottawa, ON. He provides leadership for CIHI’s information quality initiatives and for the development and application of clinical classifications and terminology standards.
 Maureen Kelly: Manager of Information Quality at CIHI, Ottawa, ON. She leads CIHI’s corporate quality program that is focused on enhancing the quality of CIHI’s data sources and information products and to fostering CIHI’s quality culture.
 Dr. Cathy Eastwood: Scientific Manager, Associate Director of Alberta SPOR Methods & Development Platform, Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, AB. She has expertise in clinical data collection, evaluation of local and systemic data quality issues, disease classification coding with ICD-10 and ICD-11.
 Dr. Hude Quan: Professor, Community Health Sciences, Cumming School of Medicine, University of Calgary, Director Alberta SPOR Methods Platform; Co-Chair of Hypertension Canada, Co-Chair of Person to Population Health Collaborative of the Libin Cardiovascular Institute in Calgary, AB. He has expertise in assessing, validating, and linking administrative data sources for conducting data science research including artificial intelligence methods for evaluating and improving data quality.
 Intended Outcomes:“What is quality health data?” The panel of experts will address this common question by discussing how to define high quality health data, and measures being taken to ensure that they are available in Canada. Optimizing the quality of clinical-administrative data, and their use-value, first requires an understanding of the processes used to create the data.
 Subsequently, we can address the limitations in data collection and use these data for diverse applications. Current advances in digital data collection are providing more solutions to improve health data quality at lower cost.
 This panel will describe a number of quality assessment and improvement initiatives aimed at ensuring that health data are fit for a range of secondary uses including data linkage. It will also discuss how the need for the linkage and integration of data sources can influence the views of the data source’s fitness for use.
 CIHI content will include:
 
 Methods for optimizing the value of clinical-administrative data
 CIHI Information Quality Framework
 Reabstraction studies (e.g. physician documentation/coders’ experiences)
 Linkage analytics for data quality
 
 University of Calgary content will include:
 
 Defining/measuring health data quality
 Automated methods for quality assessment and improvement
 ICD-11 features and coding practices
 Electronic health record initiatives

Taking a byte out of big data

With the diversification of applications and the diversity of users, the era of big data is always producing various new types of data, such as graph data, time series data, and spatial data. When predicting the future trend of the system, we always need to learn from the historical working state, and time series data plays a huge value in this aspect. Hardware resources are becoming cheaper, computing power has made major breakthroughs, and multi-machine and multi-core scenarios have become commonplace. At present, the focus of large-scale time series data processing is either focused on data collection and storage, or on single-machine processing, which has been unable to meet the existing needs. In response to this problem, this paper designs and implements a distributed large-scale time series processing and analysis system based on the Spark platform. The system framework is mainly divided into storage layer, operator layer and algorithm layer. At the storage layer, the system completes the organization and indexing of large-scale time series data based on HDFS and Hive. At the operator layer, the system provides users with basic operations commonly used for time series data on the Spark platform, and allows users to directly use these operators to implement custom time series related processing algorithms. At the algorithm level, the system implements some commonly used time series analysis algorithms in the Spark platform, including time series similarity query, clustering and prediction, so that users can directly use these algorithms to analyze time series. Finally, the feasibility and practicability of this system are verified by testing the performance and function of the system.

Although a data processing system often works as a batch processing system, many enterprises deploy such a system as a service, which we call the service-oriented data processing system. It has been shown that in-memory data processing systems suffer from serious memory pressure. The situation becomes even worse for the service-oriented data processing systems due to various reasons. For example, in a service-oriented system, multiple submitted tasks are launched at the same time and executed in the same context in the resources, compared with the batch processing mode where the tasks are processed one by one. Therefore, the memory pressure will affect all submitted tasks, including the tasks that only incur the light memory pressure when they are run alone. In this paper, we find that the reason why memory pressure arises is because the running tasks produce massive long-living data objects in the limited memory space. Our studies further reveal that the long-living data objects are generated by the API functions that are invoked by the in-memory processing frameworks. Based on these findings, we propose a method to classify the API functions based on the memory usage rate. Further, we design a scheduler called MURS to mitigate the memory pressure. We implement MURS in Spark and conduct the experiments to evaluate the performance of MURS. The results show that when comparing to Spark, MURS can 1) decrease the execution time of the submitted jobs by up to 65.8%, 2) mitigate the memory pressure in the server by decreasing the garbage collection time by up to 81%, and 3) reduce the data spilling, and hence disk I/O, by approximately 90%.

SummaryAccurate prediction of the solubility of chemical substances in solvents remains a challenge. The sparsity of high-quality solubility data is recognized as the biggest hurdle in the development of robust data-driven methods for practical use. Nonetheless, the effects of the quality and quantity of data on aqueous solubility predictions have not yet been scrutinized. In this study, the roles of the size and the quality of data sets on the performances of the solubility prediction models are unraveled, and the concepts of actual and observed performances are introduced. In an effort to curtail the gap between actual and observed performances, a quality-oriented data selection method, which evaluates the quality of data and extracts the most accurate part of it through statistical validation, is designed. Applying this method on the largest publicly available solubility database and using a consensus machine learning approach, a top-performing solubility prediction model is achieved.

BackgroundDue to recent advances in data storage and sharing for further data processing in predictive toxicology, there is an increasing need for flexible data representations, secure and consistent data curation and automated data quality checking. Toxicity prediction involves multidisciplinary data. There are hundreds of collections of chemical, biological and toxicological data that are widely dispersed, mostly in the open literature, professional research bodies and commercial companies. In order to better manage and make full use of such large amount of toxicity data, there is a trend to develop functionalities aiming towards data governance in predictive toxicology to formalise a set of processes to guarantee high data quality and better data management. In this paper, data quality mainly refers in a data storage sense (e.g. accuracy, completeness and integrity) and not in a toxicological sense (e.g. the quality of experimental results).ResultsThis paper reviews seven widely used predictive toxicology data sources and applications, with a particular focus on their data governance aspects, including: data accuracy, data completeness, data integrity, metadata and its management, data availability and data authorisation. This review reveals the current problems (e.g. lack of systematic and standard measures of data quality) and desirable needs (e.g. better management and further use of captured metadata and the development of flexible multi-level user access authorisation schemas) of predictive toxicology data sources development. The analytical results will help to address a significant gap in toxicology data quality assessment and lead to the development of novel frameworks for predictive toxicology data and model governance.ConclusionsWhile the discussed public data sources are well developed, there nevertheless remain some gaps in the development of a data governance framework to support predictive toxicology. In this paper, data governance is identified as the new challenge in predictive toxicology, and a good use of it may provide a promising framework for developing high quality and easy accessible toxicity data repositories. This paper also identifies important research directions that require further investigation in this area.

Towards a content agnostic computable knowledge repository for data quality assessment

The process of merging data of different data sources is referred to as record linkage. A medical environment with increased preconditions on privacy protection demands the transformation of clear-text attributes like first name or date of birth into one-way encrypted pseudonyms. When performing an automated or privacy preserving record linkage there might be the need of a binary classification deciding whether two records should be classified as the same entity. The classification is the final of the four main phases of the record linkage process: Preprocessing, indexing, matching and classification. The choice of binary classification techniques in dependence of project specifications in particular data quality has not extensively been studied yet. The aim of this work is the introduction and evaluation of an automatable semi-supervised binary classification system applied within the field of record linkage capable of competing or even surpassing advanced automated techniques of the domain of unsupervised classification. This work describes the rationale leading to the model and the final implementation of an automatable semi-supervised binary classification system and the comparison of its classification performance to an advanced active learning approach out of the domain of unsupervised learning. The performance of both systems has been measured on a broad variety of artificial test sets (n = 400), based on real patient data, with distinct and unique characteristics. While the classification performance for both methods measured as F-measure was relatively close on test sets with maximum defined data quality, 0.996 for semi-supervised classification, 0.993 for unsupervised classification, it incrementally diverged for test sets of worse data quality dropping to 0.964 for semi-supervised classification and 0.803 for unsupervised classification. Aside from supplying a viable model for semi-supervised classification for automated probabilistic record linkage, the tests conducted on a large amount of test sets suggest that semi-supervised techniques might generally be capable of outperforming unsupervised techniques especially on data with lower levels of data quality.

Data quality is paramount in data-driven decision-making processes, especially when dealing with large volumes of data in environments like data warehouses and data lakes. These systems store vast amounts of raw and processed data from multiple sources, making data management and quality assurance complex yet critical. With the growing adoption of Artificial Intelligence (AI), new techniques and tools have emerged that can significantly enhance data quality. This paper discusses how AI can improve the quality of data within both data warehouses and data lakes by automating data cleansing, validation, anomaly detection, and ensuring consistency. It explores the benefits, challenges, and methodologies for integrating AI tools into these systems. Keywords: Data quality, AI in data quality, data warehouse, data lakes, big data, data processing, data cleansing, data profiling.

The increasing availability of digitized biodiversity data worldwide, provided by an increasing number of institutions and researchers, and the growing use of those data for a variety of purposes have raised concerns related to the "fitness for use" of such data and the impact of data quality (DQ) on the outcomes of analyses, reports, and decisions. A consistent approach to assess and manage data quality is currently critical for biodiversity data users. However, achieving this goal has been particularly challenging because of idiosyncrasies inherent in the concept of quality. DQ assessment and management cannot be performed if we have not clearly established the quality needs from a data user’s standpoint. This paper defines a formal conceptual framework to support the biodiversity informatics community allowing for the description of the meaning of "fitness for use" from a data user’s perspective in a common and standardized manner. This proposed framework defines nine concepts organized into three classes: DQ Needs, DQ Solutions and DQ Report. The framework is intended to formalize human thinking into well-defined components to make it possible to share and reuse concepts of DQ needs, solutions and reports in a common way among user communities. With this framework, we establish a common ground for the collaborative development of solutions for DQ assessment and management based on data fitness for use principles. To validate the framework, we present a proof of concept based on a case study at the Museum of Comparative Zoology of Harvard University. In future work, we will use the framework to engage the biodiversity informatics community to formalize and share DQ profiles related to DQ needs across the community.

The collection and use of patient health data are central to any kind of activity in the health care system. These data may be produced during routine clinical processes or obtained directly from the patient using patient-reported outcome (PRO) measures. Although efficiency and other reasons justify data availability for a range of potentially relevant uses, these data are nearly always collected for a single specific purpose. The health care literature reflects this narrow scope, and there is limited literature on the joint use of health data for daily clinical use, clinical research, surveillance, and administrative purposes. The aim of this paper is to provide a framework for discussing the efficient use of health data with a specific focus on the role of PRO measures. PRO data may be used at an individual patient level to inform patient care or shared decision making and to tailor care to individual needs or group-level needs as a complement to health record data, such as that on mortality and readmission, in order to inform service delivery and measure the real-world effectiveness of treatment. PRO measures may be used either for their own sake, to provide valuable information from the patient perspective, or as a proxy for clinical data that would otherwise not be feasible to collect. We introduce a framework to analyze any health care activity that involves health data. The framework consists of four data processes (patient identification, data collection, data aggregation and data use), further structured into two dichotomous dimensions in each data process (level: group vs patient; timeframe: ad hoc vs systematic). This framework is used to analyze various health activities with respect to joint use of data, considering the technical, legal, organizational, and logistical challenges that characterize each data process. Finally, we propose a model for joint use of health data with data collected during follow-up as a base. Demands for health data will continue to increase, which will further add to the need for the concerted use and reuse of PRO data for parallel purposes. Repeated and uncoordinated PRO data collection for the same patient for different purposes results in misuse of resources for the patient and the health care system as well as reduced response rates owing to questionnaire fatigue. PRO data can be routinely collected both at the hospital (from inpatients as well as outpatients) and outside of hospital settings; in primary or social care settings; or in the patient’s home, provided the health informatics infrastructure is in place. In the future, clinical settings are likely to be a prominent source of PRO data; however, we are also likely to see increased remote collection of PRO data by patients in their own home (telePRO). Data collection for research and quality surveillance will have to adapt to this circumstance and adopt complementary data capture methods that take advantage of the utility of PRO data collected during daily clinical practice. The European Union’s regulation with respect to the protection of personal data—General Data Protection Regulation—imposes severe restrictions on the use of health data for parallel purposes, and steps should be taken to alleviate the consequences while still protecting personal data against misuse.

In this study, the application of DevOps practices in cloud-based analytics platforms indicates a way to enhance the performance of large-scale data processing systems. This integration would positively affect increased agility, scalability, and efficiency due to the implementation of continuous integration, automated deployment, and consistent monitoring. Also explored is how these practices enhance reliability in operations and adapt to changing business landscapes. The study confirms that the operational metrics gained significant improvements, such as throughput, uptime, downtime, and resource utilization, offering a big capacity and performance boost to data operations. These results illustrate the enormous impact and value of DevOps in complex data environments, thus encouraging the application of DevOps as a key strategy in contemporary data management. Keywords— DevOps Practices, Cloud-Based Analytics, Data Processing Systems, Operational Efficiency, Continuous Integration, Automated Deployment

The Multi-Azimuth UV Imager (MAVI) is a new generation atmospheric sounder carried on Tiangong-2 Space Laboratory. It can obtain the vertical distribution information of the ultraviolet radiation of the global middle and upper atmosphere, and also can detect the ultraviolet radiation of the entire atmosphere in the range of ±5° from the subastral point at the same time. The design of the instrument is unique in the world. So it is different from the traditional optical remote sensing instrument in data processing and quality analysis. In this paper, the data characteristics and data preprocessing flow of MAVI are introduced, and the data quality on-orbit of MAVI is preliminarily analyzed by using several data quality evaluation indexes. The results show that the imaging performance of MAVI is stable and the data quality is reliable. The data products of MAVI can provide effective support for atmospheric scientific research and atmospheric environment monitoring.

The Second International Workshop on Principles of Engineering Service-Oriented Systems (PESOS 2010) was held at the International Conference on Software Engineering, ICSE 2010 on May 1 and 2, 2010. PESOS 2010 provided a forum for presenting and discussing current work and research topics related to serviceoriented systems. The workshop had keynotes on SOA testing challenges and adaptive service-oriented systems. There were four paper sessions on the topics of service development, testing and evolution of service-oriented systems, service adaptation, and quality of service (QoS) and Service-Level Agreements (SLAs) in service-oriented environments. General discussions focused on these overall themes. These discussions resulted in the identification of research challenges for the future.

Quality-informed semi-automated event log generation for process mining