Clinical Pharmacology Quality Assurance Program

Comprehensive assessment of antimalarial drug resistance should include measurements of antimalarial blood or plasma concentrations in clinical trials and in individual assessments of treatment failure so that true resistance can be differentiated from inadequate drug exposure. Pharmacometric modeling is necessary to assess pharmacokinetic-pharmacodynamic relationships in different populations to optimize dosing. To accomplish both effectively and to allow comparison of data from different laboratories, it is essential that drug concentration measurement is accurate. Proficiency testing (PT) of laboratory procedures is necessary for verification of assay results. Within the Worldwide Antimalarial Resistance Network (WWARN), the goal of the quality assurance/quality control (QA/QC) program is to facilitate and sustain high-quality antimalarial assays. The QA/QC program consists of an international PT program for pharmacology laboratories and a reference material (RM) program for the provision of antimalarial drug standards, metabolites, and internal standards for laboratory use. The RM program currently distributes accurately weighed quantities of antimalarial drug standards, metabolites, and internal standards to 44 pharmacology, in vitro, and drug quality testing laboratories. The pharmacology PT program has sent samples to eight laboratories in four rounds of testing. WWARN technical experts have provided advice for correcting identified problems to improve performance of subsequent analysis and ultimately improved the quality of data. Many participants have demonstrated substantial improvements over subsequent rounds of PT. The WWARN QA/QC program has improved the quality and value of antimalarial drug measurement in laboratories globally. It is a model that has potential to be applied to strengthening laboratories more widely and improving the therapeutics of other infectious diseases.

Most proficiency testing (PT) programs operate with an open design in which clearly identified performance samples are distributed directly to participating laboratories on a shipping schedule announced in advance. In this study, we examine the effectiveness of assessing clinical laboratory performance for blood lead with an open PT by comparing its results with a double-blinded testing protocol. Aliquots from up to 72 blood lead performance pools from the New York State Department of Health and the Wisconsin State Laboratory of Hygiene were disguised as routine patient specimens and submitted in two phases to up to 42 certified clinical laboratories for blood lead analysis. These 42 laboratories also received aliquots of the same performance samples for blood lead analysis directly from the "open" PT program provider. Data reported under blind and open strategies were scored against acceptable target ranges using the Clinical Laboratory Improvement Amendments of 1988 (CLIA '88) criteria established for blood lead, i.e., +/- 0.19 micromol/L (+/- 4 microg/dL) or +/- 10%, whichever is greater. Performance differences between the strategies were also assessed. We found that 17.7% of all blind PT results were classified as unacceptable compared with only 4.5% of open PT results (P <0.001). In phase 1, 13 of 22 laboratories (60%) exhibited a statistically significant difference (P <0.05) between their blind and open PT performances, although for 6 laboratories the poorer blind performance may not necessarily have led to unsuccessful PT participation under CLIA '88 criteria. Seven (32%) laboratories had unsuccessful aggregate performance (<80%) under blind testing while maintaining successful performance in open testing. Of these seven, two had gross discrepancies motivating further investigation. The data suggest that although approximately 60% of clinical laboratories make special efforts to improve analytical performance on open PT samples relative to performance achieved for routine patient specimens, in most cases the differences are clinically insignificant and would not likely affect cumulative PT performance. Occasional use of blind PT may deter the inclination to treat performance samples more carefully.

A proficiency testing (PT) programme on the determination of benzoic acid in orange juice was organised by the Chemical Metrology Laboratory, Health Sciences Authority, Singapore, for the local food testing laboratories. The PT programme used a metrologically traceable assigned value determined by isotope dilution mass spectrometry to assess the performance of the participating laboratories. This paper discusses the reasons and approach to applying metrological principles to determine the assigned value of a PT programme. The procedure undertaken by the Laboratory to prepare a batch of PT sample, conduct homogeneity and stability testings, as well as value assignment, is presented. The discussions also include the estimation of the measurement uncertainties arising from the characterisation, testing for homogeneity and testing for stability of the PT sample. The Coordinator of the PT programme used the assigned value and its associated measurement uncertainty, together with those of the participating laboratories to assess their performances. Statistical tools applied in the performance evaluations are also presented.

South Korea’s Ministry of Food and Drug Safety (MFDS) has been developing programs for the inspection and accreditation of food sanitation inspection institutions. Food sanitation inspection institutions such as MFDS regional offices, the Research Institute of Public Health and Environment and authorized private service providers in South Korea must participate in proficiency testing (PT) programs to comply with the Food Sanitation Act and MFDS Notification No. 2012-112. As the PT provider, the MFDS annually plans various microbiological and chemical PT programs for foods, cosmetics, and pharmaceutical products in accordance with ISO/IEC 17043. The aim of this project was to evaluate the performance of microbiological PT programs to ensure the quality of their routine test results. The test materials used were freeze-dried BioBalls from BTF Pty Ltd. Homogeneity and stability were investigated to assess the adequacy of the selected test materials. This project also contains data from inter-laboratory comparisons organized by MFDS in 2011 and 2012. More than 50 laboratories attended the PT program and submitted their results. Laboratory results were rated with z-scores according to the international standard ISO 13528. The results from 2011 and 2012 revealed that all participating laboratories had similar levels of proficiency. Most of the participants received a rating of “Satisfactory.” Moreover, the percentage of participants who received a rating of “Unsatisfactory” decreased from 3.5 % in 2011 to 2.0 % in 2012.

The first proficiency testing round 630-IL-1002, was carried out with a Reference Material DMR-180a with reference values obtained by using gas chromatography isotope dilution mass spectrometry methods, in which glucose, cholesterol and creatinine were measured. The serum pool was obtained from blood donors and all the analytes were at the normal concentration in Mexican population. The laboratories participants used different field methods to measure the analytes. The Mexican compulsory standard NOM-064-SSA1-1993 “specifications for equipments in vitro diagnostic (IVD)” requests 5% precision and 5% maximum bias of the IVD equipments in the measurements of analytes like glucose and cholesterol. The results obtained by field laboratories in the proficiency testing round are compared to the reference value and uncertainty provided by the National Metrology Institute (CENAM). The quality of measurements is dependent not only on the laboratory competence but also on the methods used by those commercially available IVD kits. It is concluded that quality assessment of measurements in clinical laboratories should be critically evaluated by using stable and certified reference materials.

The validation of diagnostic methods (and the subsequent results generated by a laboratory) are improved through participation in inter-laboratory comparisons (IC), such as proficiency-testing (PT) programmes and other exercises referred to as 'ring tests' or 'ring trials' (RTs). This is a requirement to comply with international quality standards. Validating a method is a continuous process and taking part in ongoing PT programmes supports the management of a method's life cycle, providing continuing assessment of fitness (sometimes referred to as the 'validation retention status'). Proficiency-testing panel designs ensure that the methods used, particularly diagnostic specificity and sensitivity, are suitably challenged. Appraising PT results over time can illustrate whether the laboratory's performance is stable, improving or worsening, and proficiency tests can also highlight variations in the performance of assays. The development of new proficiency tests can support the implementation of novel diagnostics technologies, such as whole genome sequencing and point-of-care testing, and assist in cross-sectoral partnerships focusing on One Health approaches, which are high on the agenda for infectious disease control. For example, the rapid design and distribution of emergency exempted assays for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) means that these assays were not as rigorously evaluated as assays for established infectious diseases. Therefore, participation in PT programmes for SARS-CoV-2 is essential to understand the performance of these assays. While other mechanisms help to underpin laboratory activities, PT has been, and should remain, an integral part of laboratory quality assurance. Resources must be directed towards increasing and improving the quality of PT (for example, availability and accessibility of suitable biological and reference materials are essential for a PT provider to execute its duties), to support established and novel methods such as genomic and point-of-care tests.

Asia-Pacific Metrology Program (APMP) and Asia-Pacific Laboratory Accreditation Cooperation (APLAC) joint proficiency testing with metrological reference values for hazardous elements in cabbage

The purposes of this study are: (1) to determine whether proficiency analytical test (PAT) materials from the American Industrial Hygiene Association can be used to provide quality data for portable X-ray fluorescence analysis (XRF) of lead in dust wipe surface samples; (2) to provide data to determine whether the on-site analysis of field dust wipe samples by XRF and the laboratory method of inductively coupled plasma emission analysis (ICP) are comparable; and (3) to determine if differences exist between different wipe materials. Several wipes meet the ASTM E1792 performance requirements of lead background level less than 5 microg per wipe, be only one layer thick, yield recovery rates of 80- 120% from spiked samples, remain damp throughout the sampling procedure, and do not contain aloe. The wipes used in this study were Pace Wipes, which are used for the PAT materials, and, for the field samples, Palintest Wipes, which were supplied by the instrument manufacturer, and Ghost Wipes, which are popular because they digest in hot, concentrated acid, so that chemical analysis is simplified. Twenty PAT wipe samples were obtained from four different proficiency test rounds. Surface wipe samples were taken at three different locations representing different industry types. All samples were analyzed using a portable XRF spectrometer and by ICP. Strong linear relationships were found for the analysis of wipe samples by ICP and by portable XRF. For the PAT samples, the results from the ICP and XRF analysis were not statistically equivalent, which indicates a bias in the ICP analysis. The bias was not excessive, since all ICP analyses fell within the acceptable range for the proficiency samples. The good correlation between the proficiency sample reference values and the XRF determinations is not surprising considering similar proficiency samples were used to calibrate the instrument response. Users of this portable XRF analyzer could enroll in the proficiency test program as part of their quality assurance program. For field samples, the relationship was strongest for Palintest wipes, and the values found for all three industries could be combined. However, the results from the ICP and XRF analysis were not statistically equivalent using the correction factor in the calculation algorithm as supplied with the instrument, and a new coefficient was derived. The mean relative error for the XRF analysis versus the ICP analysis was greater than 25%, such that the method falls within the realm of screening procedures. For Ghost Wipe samples, the precision was different for different industries, and the results could not be pooled. Differences between the two wipe materials may be related to the number of folds required for analysis.

Proficiency testing (PT), recognized as a quality-assurance (QA) and quality-improvement tool, also has become the cornerstone of the Health Care Financing Administration's (HCFA) regulatory strategy under the revised Clinical Laboratory Improvement Act of 1967 (CLIA '67) and the proposed Clinical Laboratory Improvement Amendments of 1988 (CLIA '88). Use of PT as a regulatory tool corrupts it for things it can do better. PT as a primary regulatory strategy has severe limitations. We explore the nature of these limitations and their implications for clinical laboratories as they impact on the long-term success of HCFA's approved regulatory PT programs in 1991 and beyond, and CLIA '88 PT, which is to be implemented in 1994.

The outcome of proficiency tests (PTs) is influenced, among others, by the evaluation procedure chosen by the PT provider. In particular for PTs on GMO testing a log-data transformation is often applied to fit skewed data distributions into a normal distribution. The study presented here has challenged this commonly applied approach. The 56 data populations from proficiency testing rounds organised since 2010 by the European Union Reference Laboratory for Genetically Modified Food and Feed (EURL GMFF) were used to investigate the assumption of a normal distribution of reported results within a PT. Statistical evaluation of the data distributions, composed of 3178 reported results, revealed that 41 of the 56 datasets showed indeed a normal distribution. For 10 datasets, the deviation from normality was not statistically significant at the raw or log scale, indicating that the normality assumption cannot be rejected. The normality of the five remaining datasets was statistically significant after log-data transformation. These datasets, however, appeared to be multimodal as a result of technical/experimental issues with the applied methods. On the basis of the real datasets analysed herein, it is concluded that the log transformation of reported data in proficiency testing rounds is often not necessary and should be cautiously applied. It is further shown that the log-data transformation, when applied to PT results, favours the positive performance scoring for overestimated results and strongly penalises underestimated results. The evaluation of the participants’ performance without prior transformation of their results may highlight rather than hide relevant underlying analytical problems and is recommended as an outcome of this study.Graphical abstract

Water quality testing is crucial for protecting public health, especially considering the number of boil water advisories annually issued across Canada that impact daily life for residents in affected areas. To overcome these challenges, the development of drinking water safety plans and accessibility to regular testing using simple, rapid, and accurate materials are necessary. However, the significance of monitoring the accuracy of environmental microbiology testing laboratories cannot be overlooked. Participation in external quality assessment programs, such as those that include proficiency testing (PT), is a necessary risk management resource that ensures the effectiveness of these testing processes. Proficiency Testing Canada (PTC), in collaboration with the Canadian Microbiological Proficiency Testing (CMPT) program based at the University of British Columbia, have implemented a drinking-water microbiology PT program since 1996. Both PTC and CMPT are ISO/IEC 17043:2010-accredited EQA providers. The drinking water program provided PT challenges to subscribing testing laboratories twice per year. Each challenge consisted of four samples containing unknown concentrations of Escherichia coli (E. coli) and Enterobacter spp. Results from participants were assessed for accuracy based on the method of testing. This cross-sectional study evaluated 150 rural and metropolitan testing sites across Canada between 2016 and 2022. Multivariable logistic regression analysis was conducted to examine the impact of different testing methods and laboratory accreditation status on the proficiency scores. This approach enabled us to assess the association between multiple independent variables and the likelihood of achieving specific proficiency scores, providing insights into how testing methods and accreditation status affect overall performance. After adjusting for rural residence, testing time, and survey year, the membrane filtration method was positively associated with the likelihood of scoring satisfactory results compared to the enzyme-substrate method (OR: 1.75; CI: 1.37-2.24), as well as accreditation status (OR: 1.47; CI: 1.16-1.85). The potential for improvement in environmental laboratory testing performance through the implementation of regulated PT in drinking water safety plans is proposed, along with the need for reliable testing methods applicable to rapid drinking water microbiology testing.

On March 11, 2020, the World Health Organization declared a pandemic caused by the recently emerged severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). This led to increased clinical testing and decentralizing of this testing from provincial health laboratories to regional and private facilities. Leveraging the results from the Canadian Laboratory Response Network's National SARS-CoV-2 Proficiency Test (PT) Program, this study compares multiple commercial and laboratory-developed nucleic acid amplification tests, assessing both sensitivity and specificity across multiple users. Each panel consisted of six blinded, contrived-clinical samples. Panels were distributed to international, provincial and territorial laboratories and subsequently to partner facilities. Participating laboratories were asked to run these sample through their respective extraction/PCR workflows and submit results to the National Microbiology Laboratory, outlining the nucleic acid extraction platform and nucleic acid amplification test employed, as well as the viral gene target and Ct values or equivalent obtained. Data were compiled for each molecular platform and gene target used. The PT schemes were deployed in May 2020, November 2020 and June 2021, resulting in 683 data sets using 37 different nucleic acid amplification tests. Over the course of three PT schemes, the average score obtained was 99.3% by participants demonstrating consistent testing between laboratories and testing platforms. This study confirmed the rapid and successful implementation of a Canadian PT Program and provided comparative analysis of the various emergency use authorized and laboratory developed tests employed for the detection of SARS-CoV-2 and demonstrated an overall 99.3% test concordance nationwide.

○ Context.-Participation in proficiency testing is required under the Clinical Laboratory Improvement Amendments of 1988. Although the primary purpose of this testing is evaluation of current laboratory performance, a major secondary benefit of such testing is postulated to be progressive improvement in laboratory performance over time as laboratories learn from their previous experiences and feedback. Objective.-To test the hypothesis that a secondary result of proficiency testing is improvement over time of laboratory performance. Design.-The performance of participants in a large proficiency testing program (EXCEL), designed for clinic and office laboratories, on a specific problematic competence, the ability to differentiate group A streptococcus from group C streptococci, was monitored during a 6-year period (1996-2001) for changes in participant performance. Interventions.-With each testing cycle, feedback on performance relative to peers and an educational discussion analyzing performance and suggesting best practices was submitted to participants. Results.-Despite consistent feedback, there was no significant change in participant performance throughout the period studied. Conclusions.-In a large, stable proficiency testing program, a significant throat culture competence, which demonstrated less than optimal performance, did not improve over time, suggesting that current utilization of proficiency testing results in laboratory improvement programs is suboptimal.

Competency assessment is an ongoing, continuous process of monitoring individuals' abilities to perform their specific job functions. A variety of methods are useful in monitoring cytology competency, including rescreening studies, descriptive monitors (abnormality rates), discrepancy rates, workload patterns, competency-based educational programs and programs using unknown slide challenges. The goal of proficiency testing (PT) is to ascertain and assess the ability of individuals beyond the particular items or challenges presented. However, cytology PT faces many challenges for implementation as it cannot duplicate normal working conditions, and there is often no gold standard to define the truth. PT is just one measure of performance and should be considered in conjunction with other quality assessment monitors. There is no consensus on the value or validity of a large-scale regulatory PT program. Any regulatory PT program should be field tested prior to implementation, and the grading system should be scientifically defensible. Scoring of performance on PT should occur in a timely fashion, and there should be an opportunity for educational feedback. The ultimate aim of both competency assessment and PT is to positively affect laboratory procedures and improve the cervical cancer screening process.

HIV rapid diagnostic tests (RDTs) are instrumental in scaling-up HIV testing services (HTS) in low-income and middle-income countries (LMICs). HIV misdiagnosis is a growing concern in the era of expanded...