Limitations of Proficiency Testing Under CLIA '67

Most proficiency testing (PT) programs operate with an open design in which clearly identified performance samples are distributed directly to participating laboratories on a shipping schedule announced in advance. In this study, we examine the effectiveness of assessing clinical laboratory performance for blood lead with an open PT by comparing its results with a double-blinded testing protocol. Aliquots from up to 72 blood lead performance pools from the New York State Department of Health and the Wisconsin State Laboratory of Hygiene were disguised as routine patient specimens and submitted in two phases to up to 42 certified clinical laboratories for blood lead analysis. These 42 laboratories also received aliquots of the same performance samples for blood lead analysis directly from the "open" PT program provider. Data reported under blind and open strategies were scored against acceptable target ranges using the Clinical Laboratory Improvement Amendments of 1988 (CLIA '88) criteria established for blood lead, i.e., +/- 0.19 micromol/L (+/- 4 microg/dL) or +/- 10%, whichever is greater. Performance differences between the strategies were also assessed. We found that 17.7% of all blind PT results were classified as unacceptable compared with only 4.5% of open PT results (P <0.001). In phase 1, 13 of 22 laboratories (60%) exhibited a statistically significant difference (P <0.05) between their blind and open PT performances, although for 6 laboratories the poorer blind performance may not necessarily have led to unsuccessful PT participation under CLIA '88 criteria. Seven (32%) laboratories had unsuccessful aggregate performance (<80%) under blind testing while maintaining successful performance in open testing. Of these seven, two had gross discrepancies motivating further investigation. The data suggest that although approximately 60% of clinical laboratories make special efforts to improve analytical performance on open PT samples relative to performance achieved for routine patient specimens, in most cases the differences are clinically insignificant and would not likely affect cumulative PT performance. Occasional use of blind PT may deter the inclination to treat performance samples more carefully.

A proficiency testing (PT) programme on the determination of benzoic acid in orange juice was organised by the Chemical Metrology Laboratory, Health Sciences Authority, Singapore, for the local food testing laboratories. The PT programme used a metrologically traceable assigned value determined by isotope dilution mass spectrometry to assess the performance of the participating laboratories. This paper discusses the reasons and approach to applying metrological principles to determine the assigned value of a PT programme. The procedure undertaken by the Laboratory to prepare a batch of PT sample, conduct homogeneity and stability testings, as well as value assignment, is presented. The discussions also include the estimation of the measurement uncertainties arising from the characterisation, testing for homogeneity and testing for stability of the PT sample. The Coordinator of the PT programme used the assigned value and its associated measurement uncertainty, together with those of the participating laboratories to assess their performances. Statistical tools applied in the performance evaluations are also presented.

South Korea’s Ministry of Food and Drug Safety (MFDS) has been developing programs for the inspection and accreditation of food sanitation inspection institutions. Food sanitation inspection institutions such as MFDS regional offices, the Research Institute of Public Health and Environment and authorized private service providers in South Korea must participate in proficiency testing (PT) programs to comply with the Food Sanitation Act and MFDS Notification No. 2012-112. As the PT provider, the MFDS annually plans various microbiological and chemical PT programs for foods, cosmetics, and pharmaceutical products in accordance with ISO/IEC 17043. The aim of this project was to evaluate the performance of microbiological PT programs to ensure the quality of their routine test results. The test materials used were freeze-dried BioBalls from BTF Pty Ltd. Homogeneity and stability were investigated to assess the adequacy of the selected test materials. This project also contains data from inter-laboratory comparisons organized by MFDS in 2011 and 2012. More than 50 laboratories attended the PT program and submitted their results. Laboratory results were rated with z-scores according to the international standard ISO 13528. The results from 2011 and 2012 revealed that all participating laboratories had similar levels of proficiency. Most of the participants received a rating of “Satisfactory.” Moreover, the percentage of participants who received a rating of “Unsatisfactory” decreased from 3.5 % in 2011 to 2.0 % in 2012.

Competency assessment is an ongoing, continuous process of monitoring individuals' abilities to perform their specific job functions. A variety of methods are useful in monitoring cytology competency, including rescreening studies, descriptive monitors (abnormality rates), discrepancy rates, workload patterns, competency-based educational programs and programs using unknown slide challenges. The goal of proficiency testing (PT) is to ascertain and assess the ability of individuals beyond the particular items or challenges presented. However, cytology PT faces many challenges for implementation as it cannot duplicate normal working conditions, and there is often no gold standard to define the truth. PT is just one measure of performance and should be considered in conjunction with other quality assessment monitors. There is no consensus on the value or validity of a large-scale regulatory PT program. Any regulatory PT program should be field tested prior to implementation, and the grading system should be scientifically defensible. Scoring of performance on PT should occur in a timely fashion, and there should be an opportunity for educational feedback. The ultimate aim of both competency assessment and PT is to positively affect laboratory procedures and improve the cervical cancer screening process.

The validation of diagnostic methods (and the subsequent results generated by a laboratory) are improved through participation in inter-laboratory comparisons (IC), such as proficiency-testing (PT) programmes and other exercises referred to as 'ring tests' or 'ring trials' (RTs). This is a requirement to comply with international quality standards. Validating a method is a continuous process and taking part in ongoing PT programmes supports the management of a method's life cycle, providing continuing assessment of fitness (sometimes referred to as the 'validation retention status'). Proficiency-testing panel designs ensure that the methods used, particularly diagnostic specificity and sensitivity, are suitably challenged. Appraising PT results over time can illustrate whether the laboratory's performance is stable, improving or worsening, and proficiency tests can also highlight variations in the performance of assays. The development of new proficiency tests can support the implementation of novel diagnostics technologies, such as whole genome sequencing and point-of-care testing, and assist in cross-sectoral partnerships focusing on One Health approaches, which are high on the agenda for infectious disease control. For example, the rapid design and distribution of emergency exempted assays for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) means that these assays were not as rigorously evaluated as assays for established infectious diseases. Therefore, participation in PT programmes for SARS-CoV-2 is essential to understand the performance of these assays. While other mechanisms help to underpin laboratory activities, PT has been, and should remain, an integral part of laboratory quality assurance. Resources must be directed towards increasing and improving the quality of PT (for example, availability and accessibility of suitable biological and reference materials are essential for a PT provider to execute its duties), to support established and novel methods such as genomic and point-of-care tests.

Asia-Pacific Metrology Program (APMP) and Asia-Pacific Laboratory Accreditation Cooperation (APLAC) joint proficiency testing with metrological reference values for hazardous elements in cabbage

○ Context.-Participation in proficiency testing is required under the Clinical Laboratory Improvement Amendments of 1988. Although the primary purpose of this testing is evaluation of current laboratory performance, a major secondary benefit of such testing is postulated to be progressive improvement in laboratory performance over time as laboratories learn from their previous experiences and feedback. Objective.-To test the hypothesis that a secondary result of proficiency testing is improvement over time of laboratory performance. Design.-The performance of participants in a large proficiency testing program (EXCEL), designed for clinic and office laboratories, on a specific problematic competence, the ability to differentiate group A streptococcus from group C streptococci, was monitored during a 6-year period (1996-2001) for changes in participant performance. Interventions.-With each testing cycle, feedback on performance relative to peers and an educational discussion analyzing performance and suggesting best practices was submitted to participants. Results.-Despite consistent feedback, there was no significant change in participant performance throughout the period studied. Conclusions.-In a large, stable proficiency testing program, a significant throat culture competence, which demonstrated less than optimal performance, did not improve over time, suggesting that current utilization of proficiency testing results in laboratory improvement programs is suboptimal.

The increase in cases of tuberculosis (TB) during 1985-1992 and the emergence of multidrug-resistant Mycobacterium tuberculosis strains led to recommendations for rapid laboratory testing to support control efforts and selection of proper therapy (1,2). Many laboratories have adopted the recommendations to use rapid acid-fast bacilli (AFB) smears, growth detection (i.e., primary culture), identification, and drug-susceptibility testing for M. tuberculosis (3). The regulations implementing the 1988 Clinical Laboratory Improvement Amendments* (CLIA) require all laboratories that perform any mycobacteriology testing to enroll in federally approved proficiency testing (PT) programs. This report summarizes information reported by the laboratories to PT programs in the United States about their practices for M. tuberculosis.

Water quality testing is crucial for protecting public health, especially considering the number of boil water advisories annually issued across Canada that impact daily life for residents in affected areas. To overcome these challenges, the development of drinking water safety plans and accessibility to regular testing using simple, rapid, and accurate materials are necessary. However, the significance of monitoring the accuracy of environmental microbiology testing laboratories cannot be overlooked. Participation in external quality assessment programs, such as those that include proficiency testing (PT), is a necessary risk management resource that ensures the effectiveness of these testing processes. Proficiency Testing Canada (PTC), in collaboration with the Canadian Microbiological Proficiency Testing (CMPT) program based at the University of British Columbia, have implemented a drinking-water microbiology PT program since 1996. Both PTC and CMPT are ISO/IEC 17043:2010-accredited EQA providers. The drinking water program provided PT challenges to subscribing testing laboratories twice per year. Each challenge consisted of four samples containing unknown concentrations of Escherichia coli (E. coli) and Enterobacter spp. Results from participants were assessed for accuracy based on the method of testing. This cross-sectional study evaluated 150 rural and metropolitan testing sites across Canada between 2016 and 2022. Multivariable logistic regression analysis was conducted to examine the impact of different testing methods and laboratory accreditation status on the proficiency scores. This approach enabled us to assess the association between multiple independent variables and the likelihood of achieving specific proficiency scores, providing insights into how testing methods and accreditation status affect overall performance. After adjusting for rural residence, testing time, and survey year, the membrane filtration method was positively associated with the likelihood of scoring satisfactory results compared to the enzyme-substrate method (OR: 1.75; CI: 1.37-2.24), as well as accreditation status (OR: 1.47; CI: 1.16-1.85). The potential for improvement in environmental laboratory testing performance through the implementation of regulated PT in drinking water safety plans is proposed, along with the need for reliable testing methods applicable to rapid drinking water microbiology testing.

On March 11, 2020, the World Health Organization declared a pandemic caused by the recently emerged severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). This led to increased clinical testing and decentralizing of this testing from provincial health laboratories to regional and private facilities. Leveraging the results from the Canadian Laboratory Response Network's National SARS-CoV-2 Proficiency Test (PT) Program, this study compares multiple commercial and laboratory-developed nucleic acid amplification tests, assessing both sensitivity and specificity across multiple users. Each panel consisted of six blinded, contrived-clinical samples. Panels were distributed to international, provincial and territorial laboratories and subsequently to partner facilities. Participating laboratories were asked to run these sample through their respective extraction/PCR workflows and submit results to the National Microbiology Laboratory, outlining the nucleic acid extraction platform and nucleic acid amplification test employed, as well as the viral gene target and Ct values or equivalent obtained. Data were compiled for each molecular platform and gene target used. The PT schemes were deployed in May 2020, November 2020 and June 2021, resulting in 683 data sets using 37 different nucleic acid amplification tests. Over the course of three PT schemes, the average score obtained was 99.3% by participants demonstrating consistent testing between laboratories and testing platforms. This study confirmed the rapid and successful implementation of a Canadian PT Program and provided comparative analysis of the various emergency use authorized and laboratory developed tests employed for the detection of SARS-CoV-2 and demonstrated an overall 99.3% test concordance nationwide.

HIV rapid diagnostic tests (RDTs) are instrumental in scaling-up HIV testing services (HTS) in low-income and middle-income countries (LMICs). HIV misdiagnosis is a growing concern in the era of expanded...

National Institute for Food Control&amp;nbsp;Proficiency testing plays a very important role in ensuring the quality of testing. Based on&amp;nbsp;the results of the proficiency testing, the managers as well as the relevant members could assess&amp;nbsp;the technical competence of the laboratory. Participating in proficiency testing is an evidence&amp;nbsp;that a laboratory is implementing a quality management program in accordance with ISO/IEC&amp;nbsp;17025. Through the proficiency testing program, the participating laboratories themselves could&amp;nbsp;evaluate their capabilities and address the causes affecting the quality of tests so as to take&amp;nbsp;measures to ensure better quality assurance. This study provides a snapshot of the testing capacity of food testing laboratories within and outside the health sector based on chemistry proficiency testing conducted by the National Institute for Food Control (NIFC) in 2017.

Among National Institutes of Health HIV Research Networks conducting multicenter trials, samples from protocols that span several years are analyzed at multiple clinical pharmacology laboratories (CPLs) for multiple antiretrovirals. Drug assay data are, in turn, entered into study-specific data sets that are used for pharmacokinetic analyses, merged to conduct cross-protocol pharmacokinetic analysis, and integrated with pharmacogenomics research to investigate pharmacokinetic-pharmacogenetic associations. The CPLs participate in a semiannual proficiency testing (PT) program implemented by the Clinical Pharmacology Quality Assurance program. Using results from multiple PT rounds, longitudinal analyses of recovery are reflective of accuracy and precision within/across laboratories. The objectives of this longitudinal analysis of PT across multiple CPLs were to develop and test statistical models that longitudinally: (1) assess the precision and accuracy of concentrations reported by individual CPLs and (2) determine factors associated with round-specific and long-term assay accuracy, precision, and bias using a new regression model. A measure of absolute recovery is explored as a simultaneous measure of accuracy and precision. Overall, the analysis outcomes assured 97% accuracy (±20% of the final target concentration of all (21) drug concentration results reported for clinical trial samples by multiple CPLs). Using the Clinical Laboratory Improvement Act acceptance of meeting criteria for ≥2/3 consecutive rounds, all 10 laboratories that participated in 3 or more rounds per analyte maintained Clinical Laboratory Improvement Act proficiency. Significant associations were present between magnitude of error and CPL (Kruskal-Wallis P < 0.001) and antiretroviral (Kruskal-Wallis P < 0.001).

The College of American Pathologists (CAP) has maintained the highest standards for laboratory medicine through education, evaluation, and certification. One form of External Quality Assurance - proficiency testing (PT) - is the centerpiece of that mission. Over 500 medical and scientific experts oversee CAP PT programs which include more than 600 tests performed by 22,000 laboratories in over 100 countries. It is the most comprehensive laboratory peer-review comparison program in the world. The CAP offers four urine sediment PT products tailored to the needs of different laboratories. Each includes three or four digital images, shipped twice a year. The program is overseen by the Hematology and Clinical Microscopy Resource Committee. Images are graded if there is 80% or greater consensus of either referee or participant laboratories. Failing laboratories must analyze the reasons for the failure, report the results, and initiate corrective action. Over the years, there has been a progressive decline in the number of errors, demonstrating that education and regulatory oversight are major contributors to improved PT performance and, by extension, patient care. The PT urine sediment image databank is a unique resource, representing the consensus of many laboratories. Participant and referee responses identify which morphologic variants are unambiguous and which are more difficult to classify. The PT challenges include discussions of disease pathophysiology and key morphologic features. This teaching component is what helps to set the CAP's program apart. The discussions formed the basis for the Color Atlas of Urinary Sediment published by the CAP in 2010.

The College of American Pathologists (CAP) has maintained the highest standards for laboratory medicine through education, evaluation, and certification. One form of External Quality Assurance - proficiency testing (PT) - is the centerpiece of that mission. Over 500 medical and scientific experts oversee CAP PT programs which include more than 600 tests performed by 22,000 laboratories in over 100 countries. It is the most comprehensive laboratory peer-review comparison program in the world. The CAP offers four urine sediment PT products tailored to the needs of different laboratories. Each includes three or four digital images, shipped twice a year. The program is overseen by the Hematology and Clinical Microscopy Resource Committee. Images are graded if there is 80% or greater consensus of either referee or participant laboratories. Failing laboratories must analyze the reasons for the failure, report the results, and initiate corrective action. Over the years, there has been a progressive decline in the number of errors, demonstrating that education and regulatory oversight are major contributors to improved PT performance and, by extension, patient care. The PT urine sediment image databank is a unique resource, representing the consensus of many laboratories. Participant and referee responses identify which morphologic variants are unambiguous and which are more difficult to classify. The PT challenges include discussions of disease pathophysiology and key morphologic features. This teaching component is what helps to set the CAP's program apart. The discussions formed the basis for the Color Atlas of Urinary Sediment published by the CAP in 2010.