Value of Information Analysis via Active Learning and Knowledge Sharing in Error-Controlled Adaptive Kriging

Geoengineering prognoses are often based on data from a limited number of investigations of soil and rock mass. There is generally a desire to reduce the uncertainty in the prognoses while minimising the investigation costs. Value of Information Analysis (VOIA) is a support for decisions regarding investigation strategies and the aim of this paper is to present methodology for VOIA that takes into account four decision alternatives where the input data could be provided by experts. The methodology will be applied in a case study where the value of information related to an investigation borehole will be calculated. The results indicate that the value of information of the borehole is low compared with the realisation costs of the investigation. It was found that models for VOIA in underground construction projects are complex but that the analysis can be simplified with extensive use of expert knowledge and calculations of the value of perfect information as a benchmark for investigation strategies.

Efforts to ensure that funded research represents "value for money" have led to increasing calls for the use of analytic methods in research prioritization. A number of analytic approaches have been proposed to assist research funding decisions, the most prominent of which are value of information (VOI) and prospective payback of research (PPoR). Despite the increasing interest in the topic, there are insufficient VOI and PPoR applications on the same case study to contrast their methods and compare their outcomes. We undertook VOI and PPoR analyses to determine the value of conducting 2 proposed research programs. The application served as a vehicle for identifying differences and similarities between the methods, provided insight into the assumptions and practical requirements of undertaking prospective analyses for research prioritization, and highlighted areas for future research. VOI and PPoR were applied to case studies representing proposals for clinical trials in advanced non-small-cell lung cancer and prostate cancer. Decision models were built to synthesize the evidence available prior to the funding decision. VOI (expected value of perfect and sample information) and PPoR (PATHS model) analyses were undertaken using the developed models. VOI and PPoR results agreed in direction, suggesting that the proposed trials would be cost-effective investments. However, results differed in magnitude, largely due to the way each method conceptualizes the possible outcomes of further research and the implementation of research results in practice. Compared with VOI, PPoR is less complex but requires more assumptions. Although the approaches are not free from limitations, they can provide useful input for research funding decisions.

This study evaluates the cost-effectiveness of tisagenlecleucel (a CAR T-cell therapy), versus blinatumomab, for the treatment of pediatric and young adult patients with relapsed/refractory acute lymphoblastic leukemia (R/R ALL) in the Irish healthcare setting. The value of conducting further research, to investigate the value of uncertainty associated with the decision problem, is assessed by means of expected value of perfect information (EVPI) and partial EVPI (EVPPI) analyses. A three-state partitioned survival model was developed. A short-term decision tree partitioned patients in the tisagenlecleucel arm according to infusion status. Survival was extrapolated to 60months; general population mortality with a standardized mortality ratio was then applied. Estimated EVPI and EVPPI were scaled up to population according to the incidence of the decision. At list prices, the incremental cost-effectiveness ratio was EUR 73,086 per quality-adjusted life year (QALY) (incremental costs EUR 156,928; incremental QALYs 2.15). The probability of cost-effectiveness, at the willingness-to-pay threshold of EUR 45,000 per QALY, was 16 percent. At this threshold, population EVPI was EUR 314,455; population EVPPI was below EUR 100,000 for each parameter category. Tisagenlecleucel is not cost effective, versus blinatumomab, for the treatment of pediatric and young adult patients with R/R ALL in Ireland (at list prices). Further research to decrease decision (parameter) uncertainty, at the defined willingness-to-pay threshold, may not be of value. However, there is a high degree of uncertainty underpinning the analysis, which may not be captured by EVPI analysis.

The concept of Value of Information (VoI) has attracted significant attentions within the civil engineering community over especially the last decade. Triggered by the increasing focus on structural health monitoring, availability of data and emerging techniques of Big Data analysis and Artificial Intelligence, important insights on how to take benefit from VoI in structural integrity management have been gained. This literature review starts out with a summary of the historical developments and contains (1) a summary of two different VoI analysis origins, (2) a compilation of existing VoI analyses research and (3) current engineering interpretations and applications of VoI in the field of civil and infrastructure engineering. VoI analysis has roots in communication theory and Bayesian decision analysis in conjunction with utility theory. Starting point is thus taken in brief introduction of these theoretical foundations, followed by a discussion on the relevant modelling aspects such as information, probability and utility modelling. A detailed review of relevant existing research is presented, divided into the following main areas: computational methods, optimal sensor placement and engineering risk management. Finally, by way of conclusion and outlook, challenges and some promising directions for VoI analysis in the field of civil and infrastructure engineering are identified.

PNS181 Exploring the Use of VALUE of Information Analysis in HTA Decision Making at Nice

Deep-vein thrombosis (DVT) can occur in surgical patients. Routine prophylaxis can be pharmacological and/or mechanical [e.g. graduated compression stockings (GCSs)]. GCSs are available in knee length or thigh length. To establish the expected value of undertaking additional research addressing the relative effectiveness of thigh-length GCSs versus knee-length GCSs, in addition to pharmacoprophylaxis, for prevention of DVT in surgical patients. Systematic review and economic model, including value of information (VOI) analysis. Randomised controlled trials (RCTs) assessing thigh- or knee-length GCSs in surgical patients were eligible for inclusion. The primary outcome was incidence of DVT. DVT complications and GCSs adverse events were assessed. Random-effects meta-analysis was performed. To draw on a wider evidence base, a random-effects network meta-analysis (NMA) was undertaken for the outcome DVT. A review of trials and observational studies of patient adherence was also conducted. A decision-analytic model was developed to assess the cost-effectiveness of thigh- and knee-length GCSs and the VOI. Twenty-three RCTs were included in the review of effectiveness. There was substantial variation between trials in terms of the patient characteristics, interventions and methods of outcome assessment. Five trials comparing knee-length with thigh-length GCSs with or without pharmacoprophylaxis were pooled; the summary estimate of effect indicated a non-significant trend favouring thigh-length GCSs [odds ratio (OR) 1.48, 95% confidence interval (CI) 0.80 to 2.73]. Thirteen trials were included in the NMA. In the base-case analysis, thigh-length GCSs with pharmacoprophylaxis were more effective than knee-length GCSs with pharmacoprophylaxis (knee vs. thigh OR 1.76, 95% credible interval 0.82 to 3.53). Overall, thigh-length stockings with pharmacoprophylaxis was the most effective treatment, with a 0.73 probability of being the most effective treatment in a new trial of all the treatments. Patient adherence was generally higher with knee-length GCSs, and patients preferred knee-length GCSs. Thigh-length GCSs were found to be cost-effective in all but the subgroup with the lowest baseline risk, although the absolute differences in costs and effects were relatively small. The expected value of perfect information ranged from £0.2M to £178.0M depending on the scenario and subgroup. The relative effect parameters had the highest expected value of partial perfect information and ranged from £2.0M to £39.4M. The value of further research was most evident in the high-risk subgroups. There was substantial variation across the included trials in terms of patient and intervention characteristics. Many of the included trials were old and poorly reported, which reduces the reliability of the results of the review. Given that the results from both the standard meta-analysis and the NMA lacked precision (CIs were wide) owing to the heterogeneous evidence base, a new definitive trial in high-risk patients may be warranted. However, the efficiency of any further research (i.e. whether this represents value for money) is dependent on several factors, including the acquisition price of GCSs, expected compliance with thigh-length GCSs wear, and whether or not uncertainty can be resolved around possible effect modifiers, as well as the feasibility and actual cost of undertaking the proposed research. This study is registered as PROSPERO CRD42014007202. The National Institute for Health Research Health Technology Assessment programme.

PMD96 - THE IMPACT OF STRUCTURAL ASSUMPTIONS ON DECISION UNCERTAINTY: A CASE STUDY ON THE PREVENTION OF NEONATAL INFECTIONS IN PRETERM BABIES

To update the previous systematic review of the use of clopidogrel in combination with aspirin for patients with non-ST-elevation acute coronary syndrome (NSTE-ACS), investigating the optimal duration of treatment and effects of withdrawal from treatment. Ten electronic databases and internet resources were searched from 2003 to February 2007, including MEDLINE, MEDLINE In-Process, EMBASE, BIOSIS, CENTRAL and CINAHL. Randomised controlled trials (RCTs) of clopidogrel plus aspirin compared with aspirin alone were used to evaluate clinical effectiveness and safety. Inclusion criteria included any comparator trial for duration of treatment studies, and any study design conducted in patients with NSTE-ACS, percutaneous coronary intervention (PCI), stroke, peripheral artery disease (PAD) or ST-elevation myocardial infarction (STEMI) for evidence of rebound on withdrawal of treatment. The existing model was updated to provide a more robust approach to evaluating the cost-effectiveness of alternative durations of clopidogrel and to assess the potential value of further research using value of information approaches. Two RCTs were included for the review of clinical effectiveness and safety. The only RCTs identified that evaluated different durations of clopidogrel treatments were conducted in patients with stroke, PAD, STEMI or PCI. Two small RCTs and one uncontrolled retrospective cohort study were identified for the review of rebound after thienopyridine withdrawal in patients with medically-treated NSTE-ACS. On broadening the criteria, five RCTs, two observational cohorts, nine case series and 33 case reports were identified in patients post-PCI, and two case series and two case reports were identified in patients with stroke, PAD or STEMI. The CURE trial reported that the proportion of patients experiencing cardiovascular death, myocardial infarction or stroke was lower in the clopidogrel group at 30 days [relative risk (RR) 0.79; 95% confidence interval (CI) 0.67-0.92] and from 30 days to 12 months (RR 0.82; 95% CI 0.70-0.95). Clopidogrel seems to be effective in reducing adverse cardiovascular events in patients with NSTE-ACS at intermediate and high risk of ischaemic events, and appears to increase the risk of bleeding when compared with aspirin in patients with intermediate risk of ischaemic events. In terms of the cost-effectiveness of alternative durations of clopidogrel, the updated model reinforced the conclusions from the earlier analysis, i.e. a policy of 12 months of clopidogrel for patients with NSTE-ACS appears to be cost-effective in both 'average' patients and higher-risk patients. The incremental cost-effectiveness (ICER) of 12 months' duration ranged from 13,380 pounds to 20,661 pounds per additional quality-adjusted life-year (QALY) across the different scenarios. For lower-risk patients, treatment beyond 3 months does not appear to be cost-effective. The ICER of 12 months' treatment with clopidogrel varied between 49,436 pounds and 58,691 pounds per QALY. Estimates of expected value of perfect information (EVPI) were higher for the combined analysis and for analysis of high-risk patients alone (between 48.69 million pounds and 108.4 million pounds at a threshold of 30,000 pounds per QALY). At a threshold of 20,000 pounds-30,000 pounds per QALY, total EVPI ranged between 3.27 million pounds and 20.38 million pounds in the lower-risk group. The review was limited by the lack of available data. There is considerable variation in the costs of uncertainty surrounding the different scenarios and populations considered. The validity of these may also be less reliable in the higher-risk groups owing to changes in clinical practice. An adequately powered, well-conducted RCT that directly compares different durations of clopidogrel treatment in patients with NSTE-ACS would ideally be required to provide more robust evidence in relation to the impact of clopidogrel withdrawal.

BackgroundDuring the early stages of the COVID-19 pandemic, there was considerable uncertainty surrounding epidemiological and clinical aspects of SARS-CoV-2. Governments around the world, starting from varying levels of pandemic preparedness, needed to make decisions about how to respond to SARS-CoV-2 with only limited information about transmission rates, disease severity and the likely effectiveness of public health interventions. In the face of such uncertainties, formal approaches to quantifying the value of information can help decision makers to prioritise research efforts.MethodsIn this study we use Value of Information (VoI) analysis to quantify the likely benefit associated with reducing three key uncertainties present in the early stages of the COVID-19 pandemic: the basic reproduction number (R_0), case severity (CS), and the relative infectiousness of children compared to adults (CI). The specific decision problem we consider is the optimal level of investment in intensive care unit (ICU) beds. Our analysis incorporates mathematical models of disease transmission and clinical pathways in order to estimate ICU demand and disease outcomes across a range of scenarios.ResultsWe found that VoI analysis enabled us to estimate the relative benefit of resolving different uncertainties about epidemiological and clinical aspects of SARS-CoV-2. Given the initial beliefs of an expert, obtaining more information about case severity had the highest parameter value of information, followed by the basic reproduction number R_0. Resolving uncertainty about the relative infectiousness of children did not affect the decision about the number of ICU beds to be purchased for any COVID-19 outbreak scenarios defined by these three parameters.ConclusionFor the scenarios where the value of information was high enough to justify monitoring, if CS and R_0 are known, management actions will not change when we learn about child infectiousness. VoI is an important tool for understanding the importance of each disease factor during outbreak preparedness and can help to prioritise the allocation of resources for relevant information.

This paper proposes a new methodology to perform value of information (VOI) analysis within a closed-loop reservoir management (CLRM) framework. The workflow combines tools such as robust optimization and history matching in an environment of uncertainty characterization. The approach is illustrated with two simple examples: an analytical reservoir toy model based on decline curves and a waterflooding problem in a two-dimensional five-spot reservoir. The results are compared with previous work on other measures of information valuation, and we show that our method is a more complete, although also more computationally intensive, approach to VOI analysis in a CLRM framework. We recommend it to be used as the reference for the development of more practical and less computationally demanding tools for VOI assessment in real fields.

Making effective management decisions is challenging in multi‐species, multi‐threat systems because of uncertainty about the effects of different threats on different species. To inform management decisions, we often monitor species to detect spatial or temporal trends that can help us learn about threatening processes. However, which species to monitor and how to monitor to inform the management of threats can be difficult to determine. Value of information (VOI) analysis is an approach for quantifying the value of monitoring to inform management decisions. We developed a novel method that applies VOI analysis to quantify the benefits of different species monitoring strategies in multi‐threat, multi‐species systems. We applied the approach to compare the effectiveness of surveillance monitoring (monitoring species without experimentation) to targeted monitoring (monitoring species with experimentation to learn about a specific threat), and how prior information drives the benefits of these two different strategies and the species to monitor. We also illustrate the approach by applying it to two contrasting case studies for monitoring and managing declining mammals in Western Australia. Our approach shows that surveillance monitoring generally provides far lower benefits than targeted monitoring for managing threats in multi‐species, multi‐threat systems under economic constraints. Our approach also informs the choice of species to monitor and which threats to manage experimentally to most improve threat management outcomes. We show that the key parameters driving these choices include: the budget available for management, prior understanding of which threats cause declines in which species, the relative cost of managing these threats, and the background probability of decline. Our new VOI approach allows the evaluation of monitoring decisions in multi‐species, multi‐threat systems in the face of uncertainty, while explicitly accounting for the improvement in management outcomes. We recommend that managers need to explicitly consider a range of decision parameters when selecting which species to monitor to inform management. Our framework provides an objective way to do this.

This paper proposes a new methodology to perform value of information (VOI) analysis within a closed-loop reservoir management (CLRM) framework. The workflow combines tools such as robust optimization and history matching in an environment of uncertainty characterization. The approach is illustrated with two simple examples: an analytical reservoir toy model based on decline curves and a water flooding problem in a two-dimensional five-spot reservoir. The results are compared with previous work on other measures of information valuation, and we show that our method is a more complete, although also more computationally intensive, approach to VOI analysis in a CLRM framework. We recommend it to be used as the reference for the development of more practical and less computationally demanding tools for VOI assessment in real fields.

How can clinical researchers quantify the value of their proposed comparative research?

Genetic differences (polymorphisms) among members of a population are thought to influence susceptibility to various environmental exposures. In practice, however, this information is rarely incorporated into quantitative risk assessment and risk management. We describe an analytic framework for predicting the risk reduction and value-of-information (VOI) resulting from specific risk management applications of genetic biomarkers, and we apply the framework to the example of occupational chronic beryllium disease (CBD), an immune-mediated pulmonary granulomatous disease. One described Human Leukocyte Antigen gene variant, HLA-DP beta 1*0201, contains a substitution of glutamate for lysine at position 69 that appears to have high sensitivity (approximately 94%) but low specificity (approximately 70%) with respect to CBD among individuals occupationally exposed to respirable beryllium. The expected postintervention CBD prevalence rates for using the genetic variant (1) as a required job placement screen, (2) as a medical screen for semiannual in place of annual lymphocyte proliferation testing, or (3) as a voluntary job placement screen are 0.08%, 0.8%, and 0.6%, respectively, in a hypothetical cohort with 1% baseline CBD prevalence. VOI analysis is used to examine the reduction in total social cost, calculated as the net value of disease reduction and financial expenditures, expected for proposed CBD intervention programs based on the genetic susceptibility test. For the example cohort, the expected net VOI per beryllium worker for genetically based testing and intervention is $13,000, $1,800, and $5,100, respectively, based on a health valuation of $1.45 million per CBD case avoided. VOI results for alternative CBD evaluations are also presented. Despite large parameter uncertainty, probabilistic analysis predicts generally positive utility for each of the three evaluated programs when avoidance of a CBD case is valued at $1 million or higher. Although the utility of a proposed risk management program may be evaluated solely in terms of risk reduction and financial costs, decisions about genetic testing and program implementation must also consider serious social, legal, and ethical factors.

With ever-increasing complexity in Upstream project planning, to ensure decision quality, the dynamics of subsurface resources need to be embedded into concept screening to maintain consistency between the production forecast and development plan. We developed a decision-driven subsurface surrogate model that encapsulates key reservoir dynamics into the machine augmented mathematical technologies for holistic decision recommendation in concept selection and development planning under uncertainties. The surrogate model replicates the essential subsurface dynamics by using a hybrid-approach that takes into accounts both reservoir simulation data and physical first principle. In addition to standalone usage on production forecast for rapid profile screening under resource uncertainties, the subsurface surrogate model is incorporated into mathematical optimization models that simultaneously consider surface network, commercial obligation and project economics etc. to provide alternative concepts under various uncertainties. Our subsurface surrogate model has been applied for decision making on gas gathering system design, field development optimization, field-management timing and sequencing, and field tie-back study etc. Results not only show the capability of surrogate models to enable large scale rapid decision screening, but also bear a close resemblance between the predicted production profiles and the reservoir simulation results when fed with the field operating strategy recommended by our decision models with surrogate dynamics. The study demonstrates the reliability of our surrogate modeling technology on ensuring decision quality and helps build business’ confidence on technology adoption. By further incorporating subsurface uncertainties into surrogate models, the decision makers are provided with: probabilistic analysis of the outcomes, value of information analysis, cost of optionality and flexibility, and holistic project outlook etc. Our decision-driven surrogate modeling technology incorporated in mathematical decision models is the first of its kind for holistic decision support in concept selection and development planning in oil and gas industry, and has full potentials in a variety of asset lines with reliable subsurface performance prediction under uncertainties.