Evaluation of the National Institute for Health and Care Research’s (NIHR) Global Health Research (GHR) Portfolio, First Phase (2016/17-2020/21) - Interim Report
The Department for Health and Social Care (DHSC) and National Institute for Health and Care Research (NIHR) established the Global Health Research (GHR) portfolio following the 2015 UK Aid Strategy. Its purpose is to support applied health research and training in low-and middle-income countries (LMICs), addressing unmet needs and strengthening research capabilities. An initial budget of £429.5m was allocated for the first phase of the portfolio (2016/17 -2020/21), complementing other Official Development Assistance (ODA) research funders and evolving over time to include 30 programmes that NIHR or external partners manage. The portfolio is diverse in terms of its health focus, geographical scope, and funding size and emphasises the NIHR's operating principles of impact, excellence, effectiveness, inclusion, and collaboration, as well as strengthening research capacity, equitable partnerships and community engagement and involvement (CEI). It has grown to encompass 17 thematic areas, operate in over 50 LMICs, and has funded 616 awards in the first phase. The GHR Portfolio's Theory of Change (ToC) outlines the NIHR's ambition to improve global health outcomes. It recognises that it may take 3-10 years for research outputs to influence policy, practice, and behaviour changes and 10-25 years for these changes to lead to strengthened health systems and increased capacity for health promotion and disease prevention. The DHSC commissioned Ecorys through the NIHR in December 2021 to undertake an evaluation of its first phase of activities (2016/17-20/21). The portfolio evaluation is delivered in four stages: inception, interim evaluation, final evaluation (due in December 2023) and a dissemination phase (due to be completed by March 2024). The evaluation utilises a theory-based approach to assess the contributions of the GHR Portfolio to the intended outputs, outcomes, and likely impact of investments. It focuses on assessing whether the portfolio is on track to deliver the expected results given the long-term nature of the research impact. The evaluation process involves testing the portfolio's ToC, including its assumptions and causal links, to determine the validity of the underlying theory using a contribution analysis (CA) approach. CA is an approach that allows recognition of the complexities of attribution, given a whole wide range of enabling and hindering factors (internal and external) producing a result. Findings are assessed against two dimensions: Strength of evidencehow well triangulated a finding is and how much evidence there is to support it. Plausibility of contribution -how likely is it that these activities and outputs will lead to the outcomes in the theory of change. Based on this, throughout the narrative, findings are expressed in terms of their strength of evidence (strong, moderate, limited, poor), but it should be noted that this relates to how much evidence the evaluators found to support a finding, rather than a judgement on the result. The evaluation framework and questions are organized around the Organisation for Economic Co-operation and Development's Development Assistance Committee (OECD-DAC) evaluation criteria, grouped into three pillars: relevance and coherence, effectiveness and efficiency, and sustainability and impact. The framework also includes questions related to adaptability and learning, community engagement and involvement (CEI), and value for money (VfM). The key methods comprise portfolio, programme and award documentation and data review; interviews with stakeholders at portfolio, programme and award levels; an online survey to all funded awards; a Social Network Analysis (SNA) to assess networks and interactions between funded institutions; and a Bibliometric Analysis (BA) to examine research collaboration to be conducted in the final evaluation phase. A purposive sampling approach to programmes and awards for in-depth review provides rich learning and insights against the Evaluation Questions (EQs). This report presents the interim findings of the evaluation of the first phase of the NIHR's GHR Portfolio. It provides early recommendations for improving the portfolio's impact and sustainability.
Read more