Map Reduce Programming for Electronic Medical Records Data Analysis on Cloud Using Apache Hadoop, Hive and Sqoop

With the explosion of data in the past decade, big data is becoming a research hotspot in the information field. Many cloud-based distributed data processing platforms have been proposed to provide efficient and cost effective solutions for big data query processing, such as Hadoop, Hive, Pig, etc. However, most of the current research works are focus on improving the performance of query processing based on the view of systematics while without considering the characteristics of queries themselves, such as the query similarity, which will cause large numbers of redundant computation, effect query execution efficiency, thus having an adverse impact on promotion of the multi-queries processing performance. To solve this problem, in this paper, we propose a Multi-queries optimization framework based on MapReduce-oriented cloud environment (Multi-Q), which utilizes the dependence between multiple queries to realize query results reuse. Firstly, a cluster-based partition algorithm called CPA has been exploited to conduct the logic partition of the search range of query workload. Secondly, a multi-queries reuse dependence graph (MRDG) construction method on the basis of the cluster-based partition results has been presented to depict the dependence between the multiple queries. Finally, a Multi-Q processing algorithm based on Multi-Q Reuse Dependence Graph has been put forward to achieve the query results reuse and improve the overall query processing performance. We evaluate our approach by deploying Multi-Q based on Hadoop in a real cloud environment, called SEU-Cloud, and conducting extensive experiments based on the standard TPC-H. The result verifies that compared with Hive, the performance of improvement is approximately 39.3% by using our Multi-Q.

We integrate literature- and data-driven task analysis methods to derive an initial task taxonomy for electronic health record (EHR) and electronic medical record (EMR) data analysis. An EHR (EMR) is a digital and longitudinal version of a patients health(medical) information and may include all key clinical events relevant to that persons health (medical) history, such as provider, demographics, progress notes, medicine, diagnosis, etc. Our goal is to arrive a task taxonomy for analyzing EHR (EMR) datasets because tasks play an important role in the design and evaluation of visualization techniques. Our method has three stages: data collection, task modelling, and task taxonomy summary. In data collection, we first survey related literature from the past two decades and extract typical tasks and corresponding data by extracting goals and scenarios of the particular work. We introduce multiple continuous relations to describe specific binary or multiple continuous relation-seeking tasks. Finally, we arrive an initial set of task types for EHR/EMR analysis that guide the design and evaluation of visualization techniques.

Objective Current study used quantitative research methods to reveal the regional medical information research in hotspots, and summarize the mainstream knowledge base to provide the reference for researchers in the field. Methods A systematic search was conducted to find publications in CNKI, Wanfang Database and PubMed. Bibliometrics method, social network analysis, co-word analysis, principal component analysis and multidimensional scaling analysis were used in this study. Results ① total 11, 482 articles published from 1984 to 2013 were screened out. The development of research publications showed an increasing trend, and the papers were distributed in 1233 types of journals. The number of authors who located in Beijing, Guangdong, Jiangsu, Shanghai, Shandong ranked top five. ② The top 59 high-frequency key words were identified that representing the hot contents in regional health information technology research field, and was used to build knowledge network map. Among them, HIS, electronic medical records, hospital information technology, information technology were at the core of this research area. ③ We summarized research and development in the field of nine major mainstream knowledge base include: research on electronic medical records, medical personnel, information systems, information security, information networks, information sharing, information integration, information technology, and information management, resident’s health record related studies; qualitative analysis and problem-oriented research; HIS and telemedicine-related research; community-related research, two-way referral related research, health care reform and cloud computing-related research, telemedicine related research and V43 related research. Conclusions The development of regional medical information research has good momentum and closely follow the international trend, but the research contents and methods are still distance from international standard.. Research mostly focused on hospital information system construction, regional research and applied research are weak and need to be further improved. Key words: Regional medical information; Bibliometrics, co-word analysis; Social network analysis; Knowledge Network; Quantitative research

High-performance processing scientific data has enabled the development of digital resources and web-based services that facilitate uses of data beyond those that may have been envisioned by the original data producers. Scientific data processing systems must handle scientific data coming from real time, high-throughput applications. Timely processing of scientific data is important and requires sufficient available resources to achieve high throughput and deliver accurate output results. Cloud Computing provides a low-priced way for small and medium sized enterprises to process scientific data. Based on Hadoop MapReduce of cloud computing, we propose the detailed procedure of scientific data processing algorithm which can improve the overall performance under the shared environment while retaining compatibility with the native Hadoop MapReduce in this paper.

Archetype-based Electronic Health Record (EHR) systems using generic reference models from e.g. openEHR, ISO 13606 or CIMI should be easy to update and reconfigure with new types (or versions) of data models or entries, ideally with very limited programming or manual database tweaking. Exploratory research (e.g. epidemiology) leading to ad-hoc querying on a population-wide scale can be a challenge in such environments. This publication describes implementation and test of an archetype-aware Dewey encoding optimization that can be used to produce such systems in environments supporting relational operations, e.g. RDBMs and distributed map-reduce frameworks like Hadoop. Initial testing was done using a nine-node 2.2 GHz quad-core Hadoop cluster querying a dataset consisting of targeted extracts from 4+ million real patient EHRs, query results with sub-minute response time were obtained.

This study aims to obtain information on: first, the obligation to create and conceal Electronic Medical Record and its juridical consequences; Secondly, due to the law of absence or error in the manufacture of Electronic Medical Records and the position of Electronic Medical Record as a tool in the theoretical transactions.The research method used is normative juridical approach method, analytical descriptive research specification, research phase is done through literature study to examine primary law material, secondary law material, and tertiary law material. Data collection techniques are conducted through document studies, conducted by reviewing documents on positive law. Furthermore, the method of data analysis is done through normative qualitative.The results of the study indicate: Legal aspects of Medical Record or Electronic Medical Record in Teurapetik Transactions related to: first, the obligation of health workers in coaching and health services to make Medical Record or Electronic Medical Record correctly and responsible for secrecy because it is the opening of Medical Record or Electronic Medical Record without With the permission of the patient having the consequences of criminal law. The absence or misuse of the Medical Record or Electronic Medical Record means that health workers may be subject to criminal, civil and administrative sanctions. Second, the position of Medical Record or Electronic Medical Record is evidence in the form of a letter (if given outside the court), and expert information (if delivered in court).

Health Information Technologies (HIT) are being deployed world- wide to improve access to individual patient information. Primarily this is through the development of electronic health records (EHR) and electronic medical records (EMR). While the proper collection of this data has reached a high level of maturity, the use and analysis of it is only in its infancy. This data contains information which can potentially improve treatment for the individual patient and for the cohort of patients suffering a similar disease. The data can also provide valuable information for broader research purposes. In this paper we discuss the research contributions we are making in improving the use and analysis of patient data. Our projects include the analysis of physiological data, the extraction of information from multi-modal data types, the linking of data stored in heterogeneous data sources and the semantic integration of data. Through these projects we are providing new ways of using health data to improve health care delivery and provide support for medical research.

Background: Patients’ understanding of their breast cancer (BC) diagnosis is important in improving treatment adherence, shared decision-making, and clinical trial matching. However, studies have reported discrepancies between electronic medical record (EMR) and patient reported information. Using data collected from a pilot study of the Outcomes4Me patient empowerment and clinical trial matching App, we analyzed concordance of patient reported disease characteristics compared to EMR data. Methods: Data was analyzed from a single institution pilot study (NCT04262518) evaluating the feasibility of introducing the Outcomes4Me app into routine BC care. Eligibility included BC patients with any subtype or stage of invasive cancer presenting with a new diagnosis or for follow-up on active therapy. We compared patient reported characteristics within a study specific survey and/or the Outcomes4Me app for stage (metastatic or not metastatic), recurrence history, hormone receptor status, HER2, and surgery history with the data recorded in the EMR. All statistics were descriptive. We conducted the same comparison between patient reported clinical characteristics among real world users of the Outcome4Me app and EMR records downloaded by that cohort of patients. Results: Between June 2020 and December 2020, 107 patients were enrolled. Baseline demographics: 90% White, 4% Black, 3% Asian; 37% with a college degree, and 43% with post college education; 66% hormone positive/HER2-, 20% HER2+, and 13% triple negative BC; 31% were stage 4. Concordance between the survey or App questionnaire and the EMR is shown in the Table. Comparing EMR and survey data, 62% of patients matched on both HER2 and HR status, and 94% of patients matched with the EMR on metastatic and recurrence status. When surgery and treatment information was included with these features, only 57% of patients matched across all these characteristics. Similar concordance was observed between the App questionnaire and EMR. Excluding the 21% of patients reporting “unsure” HER2 status improved the concordance to 85%. Overall concordance of recurrent or metastatic status was higher than for receptor status. Despite the discordance between EMR and patient-reported disease information, 97% of patients reported that they somewhat or strongly understood their cancer diagnosis. A similar pattern of concordance between the App questionnaire and EMR was observed among a real-world cohort of 636 patients using the Outcomes4Me App who provided medical record access. Conclusion: Self report of hormone receptor and HER2 status had limited concordance with the EMR, in contrast to a high degree of accuracy for self-report of metastatic disease. The limited accuracy of self-report suggests a need for improved patient education regarding their cancer characteristics and a need for caution when relying on self-report for clinical trials matching and targeted patient education. The use of a digital platform that integrates self-report with medical record access may help address these critical needs impacting patient empowerment and care. Concordance Between App Questionnaire, Study Survey and EMR (% of patients)Disease Characteristic Matching CriteriaApp v. EMR (n=85)Survey v. EMR (n=107)App Real World Cohort (n=636)HER2 Status73%66%79%HR Status80%85%80%Combined Receptor Status67%62%73%Metastatic Status94%(n=79)97%(n=98)94%Surgery History83%(n=47)95%96%(n=310)Recurrent Status98%(n=48)97%(n=98)98%(n=411)Recurrent/Metastatic Status93%(n=42)94%(n=98)98%(n=411)Receptor/Recurrent Metastatic Status and Surgery History74%(n=33)57%(n=93)70%(n=201) Citation Format: Steven J Isakoff, Eva Glieberman, Maya Said, Agnes H. Kwak, Emily A. O’Rourke, Amanda Stroiney, Laura Spring, Beverly Moy, Aditya Bardia, Nora Horick, Jeffrey Peppercorn. Accuracy of Patient Self-Reported Breast Cancer Disease Characteristics Compared to the Medical Record in a Trial of the Outcomes4MeDigital Health App [abstract]. In: Proceedings of the 2021 San Antonio Breast Cancer Symposium; 2021 Dec 7-10; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2022;82(4 Suppl):Abstract nr P4-12-08.

Nowadays, a large volume of data is generated in the form of text, voice, video, images, and sound. It is a very challenging job to handle and to get processed these different types of data. It is a very laborious process to analyze big data by using traditional data processing applications. Due to huge scattered file systems, a big data analysis is a difficult task. So, to analyze big data, a number of tools and techniques are required, Hadoop, Apache Spark, MongoDB, Cassandra are the tools used to handle big data . Keywords: Apache Spark, Cassandra, Hadoop, MongoDB

The aim of this study was to validate a specially designed electronic medical records (EMRs) questionnaire for use in Taiwan. Published reviews of national trends and EMRs characteristics from seven countries in the Americas, Oceania, Europe, and Asia were used to design the Google-based document. This questionnaire is intended for use by medical institutions, government health units, manufacturers, academic institutions, and the public, and may be administered by information officers, non-IT staff (physicians, nurses, policymakers, vendors), and even people. A modified Delphi method was employed to determine the most appropriate content for the questionnaire. In conclusion, this research demonstrated trend analysis of several national EMRs, and also identified the meaningful use of EMR and establishing governmental EMR office are the important directions for Taiwan EMRs.

A secure electronic medical record (EMR) scheme is designed by using signcrption and blind signature. This scheme is characterized with security, privacy and forgery resisting. In this scheme, doctors, patients, TSA and medical record clerks can touch the EMR. Among them, the doctors and patients who modified the EMR illegally can always be found. TSA and medical record clerks only receive the EMR signcrypted and know nothing about the original EMR.

The maintenance and sharing of electronic medical records are one of the essential tasks in the medical treatment combination. Traditional cloud-based electronic medical record storage system is difficult to realize data security sharing. The tamper resistance and traceability of blockchain technology provide the possibility for the sharing of highly sensitive medical data. This paper proposes a safe sharing scheme of stroke electronic medical records based on the consortium blockchain. The scheme adopts the storage method of ciphertext of medical records stored in the cloud and index of medical records stored on the blockchain. The privacy protection mechanism proposed in this paper innovatively combines proxy reencryption and searchable encryption which supports patient pseudoidentity search. The mechanism could achieve controllable sharing of medical records and precise search. According to the organizational characteristics of the stroke medical treatment combination, this paper proposes an improved Practical Byzantine Fault Tolerance mechanism to reach a consensus between consensus nodes. Then, the proposed scheme is analyzed and evaluated from three aspects of medical record integrity, user privacy, and data security. The results show that the scheme can not only ensure the privacy of patient identity information and private key data but also resist the tampering and deletion attacks of internal and external malicious nodes on the medical record data. Therefore, the proposed scheme is conducive to the improvement of the timeliness of stroke treatment and the safe sharing of electronic medical records in stroke medical treatment combination.

An electronic medical record(EMR)system is built at urban community health centers to optimize services,achieving information integration of outpatient medical services based on the doctors' workstation.This system has optimized service at the outpatient clinics in the following:1.Digitizing service processes for higher efficiency and service quality; 2.Using EMR templates to save doctors' time in writing medical records for more of their time in patient inquiries and checks; 3.Using LIS,PACs and ECG/EKC systems to check lab results and imaging diagnostics of patients; 4.Using the diagnostic and therapy record sub-system to check electronic records of the patients for consulting their health history;5.Using the Shanghai Medical Alliance's(SMA)shared sub-system to acquire patients' medical records at SMA medical institutions,learning their conditions for their therapies.EMR,when it is built and put into use,can help improve GP's diagnostics and therapeutics,and provide better care to the outpatients as well. Key words: Community health center; Electronic medical record; Services; Optimization

We have witnessed the fast-growing deployment of Hadoop, an open-source implementation of the MapReduce programming model, for purpose of data-intensive computing in the cloud. However, Hadoop was not originally designed to run transient jobs in which users need to move data back and forth between storage and computing facilities. As a result, Hadoop is inefficient and wastes resources when operating in the cloud. This paper discusses the inefficiency of MapReduce in the cloud. We study the causes of this inefficiency and propose a solution. Inefficiency mainly occurs during data movement. Transferring large data to computing nodes is very time-consuming and also violates the rationale of Hadoop, which is to move computation to the data. To address this issue, we developed a distributed cache system and virtual machine scheduler. We show that our prototype can improve performance significantly when running different applications.

This study analyzed GE Centricity Electronic Medical Record (EMR) data to examine the effects of body mass index (BMI) and obesity, key risk factor components of metabolic syndrome, on the prevalence of 3 chronic diseases: type II diabetes mellitus, hyperlipidemia, and hypertension. These chronic diseases occur with high prevalence and impose high disease burdens. The rationale for using Centricity EMR data is 2-fold. First, EMRs may be a good source of BMI/obesity data, which are often underreported in surveys and administrative databases. Second, EMRs provide an ideal means to track variables over time and, thus, allow longitudinal analyses of relationships between risk factors and disease prevalence and progression. Analysis of Centricity EMR data showed associations of age, sex, race/ethnicity, and BMI with diagnosed prevalence of the 3 conditions. Results include uniform direct correlations between age and BMI and prevalence of each disease; uniformly greater disease prevalence for males than females; varying differences by race/ethnicity (ie, African Americans have the highest prevalence of diagnosed type II diabetes and hypertension, while whites have the highest prevalence of diagnosed hypertension); and adverse effects of comorbidities. The direct associations between BMI and disease prevalence are consistent for males and females and across all racial/ethnic groups. The results reported herein contribute to the growing literature about the adverse effects of obesity on chronic disease prevalence and about the potential value of EMR data to elucidate trends in disease prevalence and facilitate longitudinal analyses.