OP168 The EUnetHTA Quality Management System: Development And Evaluation

PNS262 EARLY DIALOGUE WITH NICE AND EUNETHTA: A UK PERSPECTIVE IN A POLITICALLY UNCERTAIN ERA

Beginning in January 2025, all new active substances must undergo evaluation of relative clinical effectiveness through European Union (EU) joint clinical assessments (JCAs). In the absence of head-to-head data, indirect treatment comparisons (ITCs) become indispensable in meeting the numerous population, intervention, comparators and outcomes (PICO) criteria to support decision-making. This study examined ITCs in European Network for Health Technology Assessment (EUnetHTA) relative effectiveness assessments (REAs) to obtain valuable insights into their potential implications for future JCAs. The EUnetHTA website was hand-searched for REAs of pharmaceutical products between 2010 and 2021. Information on PICO, ITC methods, ITC limitations/critiques, and relative effectiveness conclusions were systematically extracted. On the basis of the final EUnetHTA critiques, suitability of ITC evidence was categorised by the current study authors as appropriate, unsuitable or unclear. Twenty-three REAs were identified. Twelve REAs included an ITC, of which six were in oncology indications. Across the REAs, 64 comparisons were required, with a median of four comparators per REA (range 1-18). In total, 25 comparisons were informed by indirect evidence; the suitability of ITCs was categorised as unclear in all but one of the 25 comparisons. Multiple analyses and ITCs were necessary to address multiple PICOs. Although most ITCs were categorised as unclear within the REAs, they were still considered appropriate to inform decision-making. The EU JCA process will most likely require health technology developers to use various ITC approaches to address the multiple PICOs requested, recognising the inherent limitations of these methodologies. Efforts to address potential challenges for EU JCA should focus on supporting JCA assessors/co-assessors and national HTA agencies in the evaluation and interpretation of ITCs to enable decision-making.

IntroductionPotential therapies and interventions for COVID-19 are emerging and developing rapidly. In a response to this public health emergency, the European Network for Health Technology Assessment (EUnetHTA) aims to support health policy in preparation for evidence-based purchasing. To monitor the emerging evidence, a new EUnetHTA product was created: Rolling Collaborative Reviews (RCRs).MethodsRCRs are living documents that are descriptive in nature, updated monthly, and centrally coordinated. They are based on the following three sources of information: (i) published randomized controlled trials (RCTs) presented as a summary of efficacy and safety data (synthesized for a network meta-analysis conducted by the Department of Epidemiology Lazio Regional Health Service, Italy); (ii) published prospective observational studies for safety results, provided by the Map of COVID-19 Evidence conducted by the Norwegian Institute of Public Health, Norway; and (iii) RCTs registered in clinical trial registries (ClinicalTrials.gov, EudraCT Register, and the ISRCTN registry). Additionally, detailed stopping and starting rules were defined.ResultsAs of November 2020, 14 RCRs were ongoing. From the initial list of RCRs, one was suspended due to lacking effectiveness and two moved on to rapid collaborative reviews due to European Medicines Agency approvals. Four RCRs are updated on a bimonthly basis due to a lack of high-quality evidence, and five new RCRs will be started because of promising clinical studies.ConclusionsRCRs can be a means of providing timely and continuous policy support, but they require a high level of coordinated effort.

The purpose of the European network for Health Technology Assessment (EUnetHTA) is to make HTA agencies collaborate sharing methods and tools thus avoiding duplication of evaluative efforts and allowing resource savings. From 2010 to 2012, the activities of the network were carried out through EUnetHTA Joint Action 1 and Work Package 4 Strand B aimed at producing two Core HTAs with two main objectives: to test the Web based Core model and the collaborative working models. Our objective in this article is to give an historical record of the Work Package activities highlighting what worked and what did not in the collaboration of researchers' groups coming from different agencies. A retrospective description of all the steps for the joint production of the two Core HTAs is provided starting from the first step of selecting technologies of common interest. Primary researchers' views on the whole process have been collected through a semi-structured telephonic interview supported by a questionnaire. Coordinators views were gathered during internal meetings and validated. Majority of respondents thought topic selection procedure was not clear and well managed. About collaborative models, small groups were seen to enable more exchange, whatever the model. According to coordinators, loss of expertise and experience during the production process, different languages, and novelty of the Online Tool were main barriers. Lessons learned from this first experience in Joint Action 1 paved the path for the collaboration in Joint Action 2, as it allowed enhancements and changes in models of collaborations and coordination.

Several countries have developed policy frameworks allowing timely access to promising health technologies on the condition that additional evidence is generated. However, an important barrier to evidence generation is the lack of structured collaboration among health technology assessment (HTA) agencies. One of the aims of Work Package 7 (WP7) of the European network for Health Technology Assessment (EUnetHTA) Project was to determine the types of structured collaboration that could facilitate evidence generation and to develop a Web-based toolkit to support such collaboration. Collaboration modalities were defined by all WP7 Partners. Initial emphasis was on information sharing. Standardized forms for information sharing were developed and tested. An information technology development phase followed with the creation of the Web-based toolkit (Web site). Three levels of collaboration were agreed on: (i) sharing information, (ii) coordinated action, and (iii) joint action. The Web site allows access to structured and standardized forms for requesting information, posting information in response to a request, and posting information spontaneously. An online database contains all of the information requested or posted. Pilot tests on twenty-one promising technologies were satisfactory. This new Web site for sharing information on evidence generation should help countries reach robust decisions on the timely adoption of promising health technologies. It will only become fully operational if EUnetHTA Partners supply relevant, accurate, and updated information, and regularly use the Web site.

IntroductionThe European Network for Health Technology Assessment (EUnetHTA) has led together, with member states, several joint actions, including Joint Clinical Assessments (JCA), under the form of project-based voluntary cooperation, which outputs and transferability of those projects in other European countries remains somehow limited. In June 2021, the European Council has reached an agreement on the European Health Technology Assessment (HTA) regulation, which is entering into force gradually. Initially limited to oncology products, then extended to orphan/advanced therapy, and after a five to eight-year additional period, it will apply to all centrally approved products. The JCA will consist of a focused scientific analysis on relative effectiveness assessment, including the health condition, technology description, clinical effectiveness, and safety. These analyses will also include information relating to the degree of certainty. We consider that the evidence appraisal might have limitations, and transferability would not be generalizable. We aim to determine the potential drivers and barriers for HTA transferability in EU4, employing the analysis of a case example where JCA was conducted.MethodsEmploying an oncology JCA, we will compare an HTA analysis conducted in EU4 countries (Germany, France, Italy, and Spain). Overview and background information on countries involved in the JCA, and EU4 HTA system will be provided, followed by HTA outcome and main evidence requirements, reimbursement outcome, and pricing agreements.ResultsStudy results supporting HTA outcomes may focus on the population assessed, the comparator considered, and uncertainty management. A conceptual adaptation about the scope of the EU JCA regulation will be discussed, to understand its potential advantages to individual HTAs in Europe and remaining gaps to effectively inform HTA or decision-making process.ConclusionsThe analysis of pricing and reimbursement outcomes can further help understand potential barriers and drivers for JCA transferability and potential areas of evidence generation requirements.

In January 2022, the EU Regulation on Health Technology Assessment (HTAR) entered into force. The HTAR provides for the centralized conduct of EU-wide joint clinical assessment (JCA) for oncology treatments (2025 onwards), and for other medicinal products incl. vaccines (2030 onwards). HTA is a multidisciplinary process aiming to disentangle the value of a health technology in comparison to other. It has been used, to different extent across EU countries, to inform vaccine price and reimbursement decisions. Nevertheless, existing assessment frameworks utilized by HTA bodies, incl. National Immunization Technical Advisory Groups (NITAGs), as well as the relative effectiveness assessment (REA) framework developed by the European Network for Health Technology Assessment (EUnetHTA), do not fully account for the specificities of vaccines. In fact, to account for the broad value of vaccines several aspects ought to be considered, including societal benefits. Attention has been paid to these topics for years by the public health community, and in particular EUPHA-HTA, and also Vaccines Europe, a specialized vaccines group within the European Federation of Pharmaceutical Industries and Associations (EFPIA), has recently released a position statement on the need to tackle vaccine specificities in future JCA. Furthermore, as the COVID-19 pandemic has highlighted, such aspects are critical for joint procurement efforts to combat the current and future pandemics. The objective of this workshop is to address the current and future role of HTA in vaccine regulation across EU countries, inform on key specificities of vaccines in respect to JCA and HTA as a whole and discuss opportunities to develop vaccine-specific methodological guidelines within the HTAR. Following an introduction, the following topics will be addressed: • Competent bodies and HTA impact on vaccine decision-making process, incl. a discussion of implementation challenges and potential impacts of the HTAR. • JCA specificities in respect to vaccines and the importance of addressing the whole value of vaccines and vaccinations. • The proposal of Vaccines Europe for guiding principles to account for vaccines specificities in JCA built on the outcomes of an ‘across-the-industry’ project incl. also non-industry experts. The presentations will be followed by a panel discussion to further explore the themes, consider next steps and address questions from the audience. Key messages • The new EU HTAR is paving the way for JCAs of several medicinal products, incl. vaccines necessitating expert guidance in terms of determining their whole value. • Vaccines specificities should be duly considered in JCAs to inform national and EU-wide decision-making for vaccination programmes, reimbursement, and pricing.

The Scottish Health Technologies Group (SHTG) produces rapid reviews on nonmedicine technologies to inform advice for decision making in the National Health Service in Scotland (NHSScotland). This study describes our experiences and lessons learned in adapting for NHSScotland two rapid Relative Effectiveness Assessments (REAs) produced as part of the European Network for Health Technology Assessment (EUnetHTA) project to test collaborative preparation and then local adaptation. The REAs were examined for their relevance to NHSScotland and the feasibility of their adaptation evaluated using a screening toolkit. Some supplementary literature searches were conducted and additional background information on epidemiology and the specific technologies was sought. To inform decision making within Scotland, it was also necessary to identify and review cost-effective analyses. Robust evidence reviews were delivered for NHSScotland. Time saved was less than anticipated, partly due to the need to add in health economic information, and partly because of attempts to supplement and update the EUnetHTA material. The preparation of an accompanying advice statement for NHSScotland enabled the inclusion of local contextual information. Collaborative production and adaptation of HTAs is feasible and would be aided by the inclusion of cost-effectiveness analyses in the original work. Agencies should develop clear processes for adapting such reviews for their context with the focus on selecting appropriate topics for adaptation, avoiding unnecessary supplementation of the original work, and ensuring local ownership of final advice.

PIN112 - HEALTH TECHNOLOGY ASSESSMENT OF THE AVAILABLE NRTI BACKBONES IN ITALY

IntroductionThere is a range of activities that health technology assessment (HTA) doers have started to improve the process of generation of required evidence for new technologies, and the alignment of regulatory and reimbursement processes that retard the access to patients to them. Different organizations call those processes early advice, early dialogue, or scientific advice to those activities.MethodsWe performed a systematic review of the activities named scientific advice (SA), early advice (EA) and early dialogue (ED). Major databases and HTA organizations were explored. The protocol and search strategy were published in PROSPERO. The selection of final articles and documents was done in pairs, and when discrepancies were found a third person resolved with the consensus of the others. A matrix was used to define the commonalities and differences of the described processes.ResultsWe initially retrieved 949 documents, after the analysis of duplications and the full text reading of the selected ones, we finally selected 39 documents and described: the type of technologies, the process, the stakeholders, the duration, the costs, and the impact. Big HTA agencies such as the Canadian Agency for Drugs and Technologies in Health (CADTH) or the National Institute for Health and Care Excellence (NICE) included EA or SA among their portfolio of activities as well as networks (European Network for HTA (EUnetHTA) or smaller agencies such as HTA Wales or Basque Office for HTA (Osteba) among others. The type of activity, the process, duration, purpose and costs differ among HTA doers.ConclusionsThere is a need to define what we meant when we are talking about SA, ED, and EA. In fact, regulators used the same processes with different purposes. Our systematic review and the lessons learnt from the European-funded SAFENMEDTECH project will propose a detailed framework that can be useful to better understanding the needs of each of the involved parties and how to make the processes involved more efficient.

ObjectivesTo achieve universal health coverage (UHC), countries must make difficult choices to optimize the use of scarce resources. There is a growing interest in using evidence-based priority setting processes, such as Health Technology Assessment (HTA), to inform these decisions. In 2020, the Palestinian Institute of Public Health (PNIPH) and the Norwegian Institute of Public Health (NIPH) initiated a pilot to test the feasibility of coproducing an HTA on breast cancer screening in the West Bank, occupied Palestinian Territory. Additionally, a secondary aim was to test whether using an adaptive HTA (aHTA) approach that searched and transferred published evidence syntheses could increase the speed of HTA production.MethodsThe applied stepwise approach to the HTA is described in detail and can be summarized as defining a core team, topic selection, and prioritization; undertaking the HTA including adaptation using tools from the European Network for HTA (EUnetHTA) and stakeholder engagement; and concluding with dissemination.ResultsThe aHTA approach was faster but not as quick as anticipated, which is attributed to (i) the lack of availability of local evidence for contextualizing findings and (ii) the necessity to build trust between the team and stakeholders. Some delays followed from the COVID-19 pandemic, which showed the importance of good risk anticipation and mitigation. Lastly, other important lessons included the ability of virtual collaborations, the value of capacity strengthening initiatives within low- and middle-income countries (LMICs), and the need for early stakeholder engagement. Overall, the pilot was successfully completed.ConclusionThis was the first HTA of its kind produced in Palestine, and despite the challenges, it shows that HTA analysis is feasible in this setting.

In the context of strict regulatory requirements and the desire for continuous improvement, pharmaceutical companies are faced with the need to improve the efficiency and reliability of their quality management systems (QMS). Traditional approaches to the audit of QMS documentation, based on manual analysis, are time-consuming, subject to the human factor and do not always provide the necessary depth of analysis FarBrain, which uses artificial intelligence (AI) technologies for automated audit of QMS documents. The system is designed to analyze various types of documents, including standard operating procedures (SOPs), corrective and preventive action plans (CAPAs), quality policies and reports, for compliance with a wide range of international and national standards, such as EAEU GMP, ICH Q10, ISO 9001, PIC/S Guide, GAMP 5 and FDA 21 CFR Part 11. The system is based on natural language processing (NLP) and machine learning (ML) algorithms that automatically extract key information, identify inconsistencies and potential risks, and make recommendations for improvement. The article describes in detail the methods behind the operation of the system, including the architecture, the AI algorithms used, and the validation process in accordance with the principles of GMP. In addition, the results demonstrating the functionality of the system are presented and compared with other existing AI solutions in the field of quality management. The advantages, challenges and prospects of introducing AI into QMS audit processes are discussed, emphasizing the importance of integration with other digital systems and the development of predictive analytics to ensure a high level of quality and safety of pharmaceutical products.

Background: The development of technology, both hardware and software, is rapidly advancing due to the growth of Artificial Intelligence (AI), which simplifies many human activities, including the software testing process. This research journal discusses how software testing is conducted through a Quality Management System (QMS). Objective: The journal examines whether software testing processes are related to QMS, how significantly QMS supports software testing, and the differences between testing with QMS and without QMS. Methods: This study outlines the stages of software testing through QMS, such as Software Quality Assurance (SQA), Quality Control (QC), Test Maturity Model Integration (TMMI), and performance indicators. According to the American Society for Quality (ASQ), QMS is a formal and documented system consisting of policies, processes, and procedures designed to achieve an organization's quality policies and objectives. Results: Several experts also define QMS as a system that focuses on improving customer satisfaction, process consistency, efficiency, and the overall quality of products or services. Software, according to Roger S. Pressman, is a set of instructions (programs), data structures that allow information manipulation, and related documentation describing operations and functions. Conclusion: Overall, QMS not only improves software product quality but also strengthens the testing process, making it more optimal, efficient, and aligned with organizational goals. The integration of QMS frameworks such as ISO 9001:2015, CMMI, and TMMI provides organizations with a competitive advantage by ensuring consistent delivery of high-quality software products that meet or exceed customer expectations.

The provision of high-quality educational services is a mandatory objective for Higher Education Institutions (HEIs) nowadays. Consequently, HEIs implement Quality Management System (QMS) process frameworks. A core managerial activity conducted in these QMS is decision making, which impacts the overall quality of the provided HEI's educational services. This managerial context found in HEIs demands an adequate and reliable managerial decision-making support. In this article, we elaborate a new Process-Task-Decision scheme for HEIs based on a relevant international QMS process framework (ISO 9001 IWA 2:2007), and we survey selectively DMSS and HEI literature in the 1996-2016 period. We found that DMSS have been used in a variety of Process-Task-Decision situations in HEIs from early periods to the present, but their utilization is still scarce and partially deployed. Thus, open opportunities to apply them in HEIs and relevant knowledge gaps still exist to be further researched.

IntroductionThe aim of this EUnetHTA (European Network for Health Technology Assessment) Rolling Collaborative Review on high dose vitamin D for the treatment of COVID-19 was to inform health policy at an early stage in the life cycle of therapies and to monitor ongoing studies in the format of a Living Document.MethodsThe systematic literature search was conducted in Medline, Pubmed, medRxiv, bioRxiv, arXivso, Cochrane COVID-19 Study Register, ClinicalTrials.gov, ISRCTN Registry, EU Clinical Trials Register. The first search was done in January 2021, and the last in November 2021. English and German randomized controlled studies (RCTs) investigating treatment of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infected individuals with high dose vitamin D2, D3 or their metabolites were included if examining mortality, length of hospital stay, viral burden, clinical progression, hospitalization rates, intensive care unit (ICU) admission, mechanical ventilation, quality of life or adverse events. Two reviewers independently screened search results and assessed risk of bias and certainty of evidence. One reviewer extracted study data, checked by another.ResultsOf the nine RCTs published to date, two investigate calcifediol, one calcitriol and six vitamin D3. All used different dosing regimens. Disease severity and proportion of vitamin D deficiency varied between studies. Calcifediol treated patients in one study required significantly less ICU admissions than untreated patients. Vitamin D3 in another study led to significantly more SARS-CoV-2 PCR-negative patients before day 21 than placebo. There were no other significant differences between groups. Twenty-five RCTs are ongoing, five of them with over 1,000 patients.ConclusionsThe current evidence is heterogenous regarding form and dosage of vitamin D, baseline disease severity and baseline vitamin D deficiency. There is currently no standardized/recommended level of what constitutes a (beneficial) “high dose”. Most results did not show significant differences between vitamin D treated groups and no vitamin D / placebo groups. Many of the studies are very small and certainty of evidence is predominantly low or very low.