Measurement errors, linear calibration and inferences for means

Recognition that measurement error in the independent (predictor) variable of data used for regression modeling necessitates special methods of model fitting dates back to the late 1800s. In the ensuing 100 years attention was devoted almost entirely to measurement error in linear regression models, at which time advances in computing opened the door to the nonlinear statistical modeling generally, and to the study of measurement error in nonlinear regression modeling in particular. This article contains an introduction to the problem of measurement error in regression using linear regression measurement error models to illustrate key ideas. It includes a discussion of the types of measurement error commonly studied (classical and Berkson, differential and nondifferential) and how those types arise in practice. A summary of the consequences of ignoring measurement error when analyzing data measured with error is presented, with detailed results given for simple and multiple linear measurement error models. Pointers to the literature on both linear and nonlinear measurement error models are provided.

Analyses of variables measured with error are often flawed when measurement error is ignored. An extensive literature on linear measurement error models is available by [1], and more recently, for non-linear measurement error models by [2] and [3]. We investigate problems encountered in the analysis of longitudinal change, where the difference in some outcome variable is modeled on a set of regressor variables in a linear regression model. This type of analysis has become quite popular in biomedical research, and the results are often biased when the outcome variable is measured with error and its observed baseline value is included as a covariate in the fitted model. [4] demonstrated the effects of measurement error in the analysis of change in wall thickness of the common carotid artery. They showed the naive analysis led to erroneous findings due to the measurement error bias, even when the regressor variables were assumed to be measured precisely. In this paper, we present a method-of-moments correction for measurement error bias, provided the measurement error variance is known or can be estimated. This work extends the work of [4] to include regressor variables that are measured with error.

Measurement errors are very often come upon in data analysis. Classical statistical methods become disadvantageous and ordinary least squares estimator of parameters turns into inconsistent and biased, in the existence of measurement errors in the data. Although some methods are used, the performances of them are not good enough in the presence of multicollinearity and measurement errors in the data, simultaneously. That’s why researchers have been inquiring about the estimation of the parameters if the measurement error models have multicollinearity, lately. Especially, biased estimation techniques have been researched in the existence of multicollinearity for measurement error models recently. In this paper, the ridge and Liu estimation approaches to the measurement error models in the existence of multicollinearity are investigated. The comparisons of the biased estimators’ performances are analyzed theoretically and numerically.

In this paper, we generalize the notion of measurement error on deterministic sample datasets to accommodate sample data that are random-variable-valued. This leads to the formulation of two distinct kinds of measurement error: intrinsic measurement error, and incidental measurement error. Incidental measurement error will be recognized as the traditional kind that arises from a set of deterministic sample measurements, and upon which the traditional measurement error modelling literature is based, while intrinsic measurement error reflects some subjective quality of either the measurement tool or the measurand itself. We define calibrating conditions that generalize common and classical types of measurement error models to this broader measurement domain, and explain how the notion of generalized Berkson error in particular mathematicizes what it means to be an expert assessor or rater for a measurement process. We then explore how classical point estimation, inference, and likelihood theory can be generalized to accommodate sample data composed of generic random-variable-valued measurements.

We investigated the conditions under which the information matrix, conditional on covariates, and the unconditional version, integrated over marginal distribution of covariates, are increasing functions of the measurement error variance, and when the conditional and unconditional asymptotic variances of the maximum likelihood estimate are decreasing functions of the measurement error variance. We say that a paradox occurs when one can decrease the variance of the maximum likelihood estimate by increasing measurement error. Two covariate measurement error models were considered: the Berk-son and the classical measurement error models, with continuous and binary dependent variables, The measurement error variance was assumed known. We found that in the linear model with Berkson eovariate measurement error, the paradox can occur when the model variance is known and quite large. With binary data and the Berkson eovariate measurement error, the paradox is likely to occur for rare events. The paradox based upon the unconditional variance, calculated as the inverse of the unconditional information matrix, does not occur with the classical measurement error model. However, if does occur for the conditional variance, as calculated by the inverse of the observed information matrix, in some circumstances. Empirical evidence for this is illustrated using data from the Nurses' Health Study.

Population abundances are rarely, if ever, known. Instead, they are estimated with some amount of uncertainty. The resulting measurement error has its consequences on subsequent analyses that model population dynamics and estimate probabilities about abundances at future points in time. This article addresses some outstanding questions on the consequences of measurement error in one such dynamic model, the random walk with drift model, and proposes some new ways to correct for measurement error. We present a broad and realistic class of measurement error models that allows both heteroskedasticity and possible correlation in the measurement errors, and we provide analytical results about the biases of estimators that ignore the measurement error. Our new estimators include both method of moments estimators and "pseudo"-estimators that proceed from both observed estimates of population abundance and estimates of parameters in the measurement error model. We derive the asymptotic properties of our methods and existing methods, and we compare their finite-sample performance with a simulation experiment. We also examine the practical implications of the methods by using them to analyze two existing population dynamics data sets.

Membership notes

Nonparametric errors in variables models with measurement errors on both sides of the equation

Preterm birth, defined as delivery before 37 completed weeks' gestation, is a leading cause of infant morbidity and mortality. Identifying factors related to preterm delivery is an important goal of public health professionals who wish to identify etiologic pathways to target for prevention. Validation studies are often conducted in nutritional epidemiology in order to study measurement error in instruments that are generally less invasive or less expensive than “gold standard” instruments. Data from such studies are then used in adjusting estimates based on the full study sample. However, measurement error in nutritional epidemiology has recently been shown to be complicated by correlated error structures in the study-wide and validation instruments. Investigators of a study of preterm birth and dietary intake designed a validation study to assess measurement error in a food frequency questionnaire (FFQ) administered during pregnancy and with the secondary goal of assessing whether a single administration of the FFQ could be used to describe intake over the relatively short pregnancy period, in which energy intake typically increases. Here, we describe a likelihood-based method via Markov chain Monte Carlo to estimate the regression coefficients in a generalized linear model relating preterm birth to covariates, where one of the covariates is measured with error and the multivariate measurement error model has correlated errors among contemporaneous instruments (i.e., FFQs, 24-hour recalls, and biomarkers). Because of constraints on the covariance parameters in our likelihood, identifiability for all the variance and covariance parameters is not guaranteed, and, therefore, we derive the necessary and sufficient conditions to identify the variance and covariance parameters under our measurement error model and assumptions. We investigate the sensitivity of our likelihood-based model to distributional assumptions placed on the true folate intake by employing semiparametric Bayesian methods through the mixture of Dirichlet process priors framework. We exemplify our methods in a recent prospective cohort study of risk factors for preterm birth. We use long-term folate as our error-prone predictor of interest, the FFQ and 24-hour recall as two biased instruments, and the serum folate biomarker as the unbiased instrument. We found that folate intake, as measured by the FFQ, led to a conservative estimate of the estimated odds ratio of preterm birth (.76) when compared to the odds ratio estimate from our likelihood-based approach, which adjusts for the measurement error (.63). We found that our parametric model led to similar conclusions to the semiparametric Bayesian model.

The recommended Practice API 1163 In-line Inspection Systems Qualification, published in September 2021, documents three levels of in-line inspection (ILI) measurement validation. The first level of validation, Level 1, relies on previous experience with the ILI system and adherence to operating procedures. The second level of validation, Level 2, uses field data to validate the vendor specification. The third and final level of validation, Level 3, uses advanced statistical methods to estimate the measurement performance from field measurements. The recommended practice describes methodologies for each of these levels of validation, however operators may encounter some challenges when implementing the methodologies. Some of the key challenges for operators when following the recommended practice include determining when it is acceptable to use each validation level, and how to interpret some of the guidance provided in the recommended practice. Additionally, there is no clear guidance on how the uncertainties and benefits of each decision in this process should be evaluated when applied to practical pipeline integrity assessment scenarios. This report addresses these challenges by developing detailed guidelines related to each of the three validation levels. These guidelines, which complement the recommended practice, aim to clarify the requirements outlined in the recommended practice, such as what is meant by ‘similar pipelines’ for a Level 1 validation, while also proposing new methods, such as the inclusion of prior information, that can be used to facilitate the validation process. The guidelines were developed in collaboration with several subject matter experts across the industry and are presented in a way that can easily be incorporated into an operator’s internal processes. To this end, a Microsoft Excel Spreadsheet tool was also developed as part of this project to facilitate and standardize the implementation of the recommended practice. The tool includes functionality that is reflective of the guidelines presented in this report and allows the user to perform a full measurement performance validation for any of the levels. Finally, this report presents several case studies that illustrate the use of these guidelines in real-world scenarios. Ultimately, the recommended practice is designed to be flexible, allowing pipeline operators to tailor their compliance with various approaches. The guidelines developed in this project represent one way to meet the requirements in the recommended practice, but they are not mandatory, nor do they represent the only valid methodology. The addendum summarizes additions to the Microsoft Excel Spreadsheet tool developed in project number IM-1-06 for the practical implementation of the measurement performance validation methodology outlined in the API RP 1163 2021 Edition (API 1163-2021). The addendum covers the following new features: - Crack and Dent Assessments: the spreadsheet has been updated to consider the additional data columns and attributes necessary to assess the ILI measurement performance of cracks and dents. The updates include incorporating field measurement errors, the ability to input both length and depth measurements, and handling data subsets. - Measurement Error Model: field measurement error is characterized by a ratio of the ILI to field error. The ILI is modelled as having either a systematic bias or a linear regression to true dimensions. - Subset Data: multiple subsets of data can be analyzed separately. Incorporating data sub-sets allows operators to compare the measurement performance for categories of interest. - Simplification of the Probability of Identification: POI has been simplified by assuming the field-identified anomaly type is correct. The POI is calculated for each anomaly type in the dataset. Only reported anomalies are included in the revised POI calculation. - Expansion of Level 2 and Level 3 Tests: the null hypothesis and tolerance interval estimate methods have been expanded to perform retrospective analysis of the Level 2 test and one-sided Level 3 tests. - Burst Pressure Comparison: the spreadsheet has been updated to include additional data inputs for comparing the Failure Pressure Ratio (FPR) estimates derived from ILI measurements and field measurements. The five (5) files included in this zipped deliverable package are: 1) PR719-243812-R01 Addendum to API 1163 Performance Validation Guidelines - Crack and Dent.pdf 2) PR719-223803-R01 API 1163 Performance Validation Guidelines.pdf 3) PR719-223803-M01 API 1163 Performance Validation Guidelines Spreadsheet User Guide.pdf 4) PR719-243812-E01 API 1163 Performance Validation Spreadsheet Tool.xlsx 5) PRCI Spreadsheet EULA.pdf

Microarray technology has been widely used to analyze the gene expression levels by detecting fluorescence intensity in a high throughput fashion. However, since the measurement error produced from various sources in microarray experiments is heterogeneous and too large to be ignored, we propose here a measurement error model for microarray data processing, by which the standard deviation of the measurement error is demonstrated to be linearly increased with fluorescence intensity. A robust algorithm, which estimates the parameters of the measurement error model from a single microarray without replicated spots, is provided. The model and algorithm for estimating of the parameters from a given data set are tested on both the real data set and the simulated data set, and the result has been proven satisfactory. And, combining the measurement error model with traditional Z-test method, a full statistical model has been developed. It can significantly improve the statistical inference for identifyin...

The derivation of the likelihood function for binary data from two types of main study/validation study designs where model covariates are measured with error is elaborated. Rather than limiting consideration to a restricted family of models with convenient mathematical properties, we suggest that empirical considerations, customized to the data at hand, should drive model choices. The joint likelihood function for the main study, in which the covariates are measured with error, and the validation study, in which they are not, is maximized, and estimation and inference proceeds using standard theory. Although the choice of the measurement error model is driven by empirical considerations, the relatively small validation study sizes typically seen may lead to misspecification, resulting in bias in estimation and inference about exposure-disease relationships. By using a nonparametric form for the measurement error model, the resulting semi-parametric methods suggested by Robins, Rotnitzky, and Zhao (1994, Journal of the American Statistical Association 89, 864-866) and Robins, Hsieh, and Newey (1995, Journal of the Royal Statistical Society, Series B 57, 409-424) are free from bias due to misspecification of the measurement error model, trading efficiency for robustness as usual. These fully and semi-parametric methods are illustrated with a detailed example from a main study/validation study of the health effects of occupational exposure to chemotherapeutics among pharmacists (Valanis et al., 1993, American Journal of Hospital Pharmacy 50, 455-462). A constant, prevalence ratio model for common binary events, with gamma covariate measurement error, is derived and empirically verified by the available data. A careful reanalysis of the data, taking measurement error fully into account, leads to a threefold increase in the log relative risk and no loss of statistical power. The semi-parametric estimates are consistent with the parametric results, providing reassurance that important bias due to misspecification of the measurement error model is unlikely.

This thesis contains three essays on inference in econometric models. Chapter 1 considers the question of bootstrap inference for Propensity Score Matching. Propensity Score Matching, where the propensity scores are estimated in a first step, is widely used for estimating treatment effects. In this context, the naive bootstrap is invalid (Abadie and Imbens, 2008). This chapter proposes a novel bootstrap procedure for this context, and demonstrates its consistency. Simulations and real data examples demonstrate the superior performance of the proposed method relative to using the asymptotic distribution for inference, especially when the degree of overlap in propensity scores is poor. General versions of the procedure can also be applied to other causal effect estimators such as inverse probability weighting and propensity score subclassification, potentially leading to higher order refinements for inference in such contexts. Chapter 2 tackles the question of inference in incomplete econometric models. In many economic and statistical applications, the observed data take the form of sets rather than points. Examples include bracket data in survey analysis, tumor growth and rock grain images in morphology analysis, and noisy measurements on the support function of a convex set in medical imaging and robotic vision. Additionally, nonparametric bounds on treatment effects under imperfect compliance can be expressed by means of random sets. This chapter develops a concept of nonparametric likelihood for random sets and its mean, known as the Aumann expectation, and proposes general inference methods by adapting the theory of empirical likelihood. Chapter 3 considers inference on the cumulative distribution function (CDF) in the classical measurement error model. It proposes both asymptotic and bootstrap based uniform confidence bands for the estimator of the CDF under measurement error. The proposed techniques can also be used to obtain confidence bands for quantiles, and perform various CDF-based tests such as goodness-offit tests for parametric models of densities, two sample homogeneity tests, and tests for stochastic dominance; all for the first time under measurement error.

We consider quasilikelihood models when some of the predictors are measured with error. In many cases, the true but fallible predictor is impossible to measure, and the best one can do is to obtain replicates of the fallible predictor. We consider the case that the replicates are not independent. If one assumes that replicates are independent and they are not, one typically underestimates the extent of the measurement error, leading to an inconsistent errors in variables correction. We devise techniques for estimating the measurement error covariance matrix. In addition, we discuss how one might perform a quasilikelihood analysis by computing the mean and variance functions of the observed data, both using approximations and also exactly through a Monte Carlo method. The methods are illustrated on a data set involving systolic blood pressure and urinary sodium chloride, where the measurement errors appear to be approximately normally distributed but highly correlated, and the distribution of the true predictor is reasonably modeled as a mixture of normals.

The large need for small area data and limited auxiliary information drive the development of small area estimation methods with auxiliary information comes from survey data. The consequence of the existence of the auxiliary variables from survey data is the development of measurement error models. Survey data is used as auxiliary variables that are taken randomly so that the data is considered to be stochastic. Thus, the measurement error model is assumed to be structural. Meanwhile, auxiliary information or covariates does not always have a normal distribution but sometimes contain outliers so the assumption of the t-distribution is considered to be more appropriate. Therefore, we use the moment-method to estimate the parameters and develop an empirical Bayes-EB predictor in a nested error regression model with measurement errors in the area-level covariates. In addition, the covariate in this model is assumed in the t-distribution which were previously always considered normal. Using simulation studies, we can report the performance of EB predictor under true covariates and measurement errors assumed to be t-distributions based on mean squared prediction errors (EMSPE). The results show that the model we developed leads to a significant increase in efficiency compared to EB predictors previously proposed. Furthermore, this approach is applied in National socio-economic survey (Susenas) data in Malang Regency with the aim of predicting mean years of schooling by districts using monthly per capita household expenditure data as the covariate variables that are considered to have the t-distribution.