Data Acquisition—Observation

Abstract

Translate Article

First of four chapters covering data assembly, being i) observation (predictors); ii) performance (target); iii) target definition; iv) data merge. (1) Make a plan—finalize the what, where, who and how of data, especially as regards internal and external data sources and how subject-level data will be matched. (2) Gather—identify key fields and determine what data aggregates are required, set retention rules and obtain retrospective histories from external sources. (3) Reduce—identify predictors to be dropped or ignored, especially those un- or under-populated, unpredictive, proscribed, highly correlated and … those populated after the decision was made. (4) Cleanse—individual records that are out-of-scope, underpopulated, and duplicates. Mention is also made of the treatment of outliers for continuous targets and of inconsistencies within the data {e.g. varying codes by time or source}. (5) Check—ensure file contents are consistent with expectations.