Assessing quality in the specialized hemostasis laboratory: review and critique of external quality assurance (EQA) programs in the US

Background

As the laboratory is a critical source for results that lead to the diagnosis and management of patients, the National Committee for Clinical Laboratory Standards [NCCLS, now known as Clinical & Laboratory Standards Institute (CLSI)] introduced in 1999, the concepts of quality practice for clinical laboratories (1). The premise of this concept was to improve all phases of laboratory testing, specifically pre-analytical, analytical and post-analytical processes. Hemostasis testing is particularly sensitive to pre-analytical issues, including sample collection, handling, transportation, processing and storage (2). Each of these steps needs to be addressed and controlled locally with proper, consistent institutional procedures and protocols to minimize pre-analytical errors. Quality assurance measures specific for the analytical phase of hemostasis testing occurs: (I) during the initial test evaluation (method validation of test performance); (II) as part of periodic quality control (QC) (daily or as required QC performance); and (III) during episodic assessment using external quality assurance (EQA, also commonly referred as Proficiency Testing). According to the College of American Pathologists (CAP), the benefits of participation in an EQA program include improving laboratory practice, characterizing local test performance over site specific and multiple platforms, identifying potential test interferences (e.g., medication effect on test results), helping to improve laboratories that perform poorly and thereby satisfy regulatory requirements (3). In the United States (US), the Center for Medicare & Medicaid Services (CMS) regulates all human laboratory testing using Clinical and Laboratory Improvement Amendments (CLIA) regulations. CLIA requires that all testing, including hemostasis testing, be subject to EQA, with the additional requirement that certain hemostasis tests must be enrolled in CMS-approved proficiency programs. In addition to the College of American Pathologist (CAP) EQA program, there are other international hemostasis EQA programs, including the United Kingdom National External Quality Assessment Service (UK NEQAS, https://ukneqas.org.uk), External quality Control of diagnostic Assays and Tests (ECAT, https://www.ecat.nl) from the Netherlands, Royal College of Pathologists from Australasia Quality Assurance Program (RCPA-QAP, https://rcpaqap.com.au), among others. A coalition of international thrombosis and hemostasis EQA programs has also led to the creation of the External Quality Assurance in Thrombosis and Hemostasis (EQATH, http://eqath.org) organization, dedicated to “form a working group to identify, evaluate, compare and improve existing quality assessment (EQA) programs” as well as their “overall aim of improving the quality of coagulation diagnostic testing…” (4). The feature tenet of the published EQATH article is “Do the right thing.” (4). This message conveys the notion that an EQA program may satisfy regulatory requirements; however, if the EQA samples are always normal or the EQA material is not consistent with patient testing, then merely satisfying the regulation does not assure that testing quality is enhanced or assured.

In the US, CAP proficiency testing is the primary EQA program for satisfying the requirements of CMS accreditation. In the US, CAP EQA (or CAP approved alternative EQA programs, see Table 1) is required by CMS for the following routine hemostasis tests: prothrombin time (PT), activated partial thromboplastin time (APTT), and fibrinogen (FBG). In addition, for those laboratories that are CAP accredited, the following tests must be performed using a CAP approved EQA program: International Normalized Ratio (INR), D-dimer (XDP), factor VIII activity (F8), IgG and IgM isotype anticardiolipin antibodies (ACA), and homocysteine (HCY). For all other hemostasis tests, EQA can be performed using alternative EQA programs, even though they may not be approved by CAP or CMS.

Assessing EQA: pre-analytical issues

Common pre-analytical problems associated with hemostasis testing include improper blood collection (e.g., wrong tube, wrong patient, wrong collection time, inadequate blood volume in collection tube), processing (e.g., incorrect centrifugation speed or duration, delays in processing whole blood), sample condition (lipemia, icterus, hemolysis), and storage (room temperature versus refrigerated versus frozen stability (5). In addition, the improperly ordered tests (e.g., PT for assessing unfractionated heparin anticoagulation) or inappropriate testing (e.g., testing protein C in a patient receiving warfarin) are also pre-analytical variables that present challenges to the laboratory and can lead to inaccurate results which can impact patient diagnosis and management.

Contemporary EQA programs do not address the pre-analytical variables associated with laboratory testing. Furthermore, commercial EQA samples introduce additional variables that may impact the quality of the test result. For plasma-based testing, EQA samples are commonly lyophilized, which requires strict adherence to re-dilution of sample volume and adequate mixing; samples may also be subject to reconstitution instability. The process of lyophilization may also impact the analyte being measured. Additionally, some plasma-based EQA samples are contrived such as adding substances (e.g., drugs) that are known to affect some coagulation testing, but also leading to effects that may not be predictable or universal for all analytes within the survey. Coagulation proficiency testing samples are commonly diluted with saline to provide ‘abnormal’ samples containing low levels of both coagulation factors and naturally occurring anticoagulants. Dilution however, alters the matrix of the sample to be tested. In real world patients, the contamination of saline (e.g., blood collected above intravenous line) or drugs [e.g., direct oral anticoagulants (DOAC) and thrombophilia testing] would be identified as samples most likely to be cancelled before testing, and so these contrived EQA samples may not represent actual clinical practice. For platelet function proficiency testing that is currently available, the collection of normal donor blood samples is required, with subsequent pooling of collected blood into plastic vials containing saline or anti-platelet drugs. Normal donors for EQA platelet studies are often considered to be ostensibly healthy and drug naïve for the past 7–10 days. With the exception of interferences and sample stability, the EQA associated pre-analytical variables are not found in real world patient testing.

Lastly, EQA samples are required to be processed and tested in the same manner as patient samples. While an ideal premise, this concept is fairly unrealistic given that EQA samples are often shipped as lyophilized vials, must be reconstituted and relabeled to adapt to coagulation instruments, and must be handled in a separate manner than the majority of patient samples. As EQA programs fall short of addressing pre-analytical variables associated with hemostasis testing, each laboratory should develop some mechanism for episodically assessing control of pre-analytical conditions as these variables contribute to the vast majority of testing errors (5).

Assessing EQA: analytical issues

For common hemostasis tests, there are adequate sources of EQA samples that provide a good measure for longitudinal and peer-group comparison for test accuracy. This includes the PT/INR, APTT, and D-dimer. For other analytes, it is more often challenging to obtain samples that reflect or mimic true abnormal patient samples. As such, contrived material is commonly used. Saline diluted samples could provide an adequate assessment of whether a given laboratory can differentiate normal from abnormal results. However, the use of saline diluted samples is limited for those EQA programs that assess a vast number of analytes. For example, an EQA module could contain a normal sample (usually a commercially prepared normal control or normal pooled plasma) and saline diluted sample (e.g., 1:1 saline to plasma) of the same material to achieve a reduction in coagulation factors. Such dilution changes the plasma matrix and leads to a pan reduction in coagulation factors, unlike most actual patient samples. This material (which could identify normal and abnormal factor levels), would not be adequate for assessing other tests that would not be affected by saline dilution (e.g., euglobulin lysis time or positive mixing studies).

Additionally, most EQA programs do not address the needs of special coagulation laboratories with complex esoteric test menus. These include tests such as factor inhibitor assays, heparin-induced thrombocytopenia (HIT) assays and ADAMTS-13 testing. Some of these esoteric hemostasis tests also lack commercial QC material (e.g., non-factor VIII inhibitor assays, euglobulin lysis time, platelet aggregation studies) that further adds an element of uncertainty to the test accuracy.

When EQA programs cannot provide samples that can differentiate normal from true abnormal test results, then confidence that a laboratory is providing an accurate and reproducible result is diminished. As an example, the CAP platelet aggregation module uses both contrived samples (a drug added ex vivo) and CAP provided reagents, although more recently, CAP has requested testing sites provide their own agonists. As only ADP and epinephrine agonists are used for the platelet aggregation module, this EQA program does not assess the adequacy of local laboratories reagents, nor does it reflect the spectrum of agonists (e.g., collagen, ristocetin, arachidonic acid, etc.) recommended for assessing platelet function (6). Modules for thromboelastography (TEG or rotational thromboelastogram, ROTEM) only assess selected parameters for reporting, and do not address either alternative methodologies (e.g., TEG heparinase reagent) or result interpretations. The EQA modules for these assays are often relabeled lyophilized commercial control material, which does not reflect the vast majority of patient testing using citrated or neat whole blood.

Where EQA programs do not adequately evaluate the laboratory methods, then such testing should be assessed using either alternative EQA strategies or internal QA programs. EQA programs other than CAP can be accessed in the US that can be used to provide patient samples to assess esoteric coagulation and drug assessment. Whole blood testing is challenging for commercial EQA, and thus the laboratory itself, then it becomes incumbent on the laboratory to be responsible for incorporating a mechanism for creating samples that mimic their clinical practice to assure that either reagents (e.g., heparinase TEG) or battery of results (e.g., platelet aggregation studies) reflecting expected patient outcome. For TEG or ROTEM parameters that are not EQA assessed (e.g., lysis), the addition of a thrombolytic agent to a normal sample should yield expected lysis results in these methods. The TEG heparinase reagent can bind up to 5 U/mL of unfractionated heparin and is used to assess patients coming off cardiopulmonary bypass. Creating citrated whole blood samples that contain both UFH naïve and up to 5 U/mL would determine whether that reagent is performing as expected, and both the UFH naïve and UFH treated sample tested with heparinase reagent should yield similar results. For platelet aggregation studies, the ex-vivo addition of anti-platelet agents (e.g., aspirin, prasugrel, platelet glycoprotein IIb-IIIa blockers) would mimic expected aspirin, P2Y12 inhibitor and Glanzmann’s thrombasthenia-like results, respectively.

EQA: post-analytical issues

The components associated with post-analysis usually include result reporting, reference interval comparison, and in some cases, result interpretation or an interpretative report for a battery of tests or a multi-component test (7). For result reporting, most laboratories have a laboratory information system (LIS) that receives and reports laboratory data either electronically directly from the instrument, or manually entered. EQA programs can assess the competence of an institutions protocol for manually inputting test results, as these EQA programs require conversion of reported results into the EQA website using manual keyboard entry from a computer. For some EQA modules and some analytes within a module, there are sections for interpretation, but these are typically limited to either “normal” or “abnormal”. However, these comprehensive modules, while satisfying a regulatory requirement, do not reflect clinical practice which would look at all the tests in totality and not just an individual result, in order to provide an interpretation of all the reported data to a clinician. This is especially significant in cases where thrombophilia tests are performed on patients on oral anticoagulants, interpretation of platelet aggregation studies, interpretation of heparin induced thrombocytopenia results in the context of clinical pretest probability scores, and others. Additionally, there are age adjusted reference ranges for reported results, which would impact whether a low factor IX in a neonate would be considered normal whereas in an adult patient would be considered to be significantly low. A majority of contrived samples, if received by the laboratory as patient sample, would be rejected outright for testing, and certainly would be addressed as a potential cause of the reported abnormal result.

Lastly post-analytical considerations are the reported results and conclusions from the EQA provider. A recent publication highlighted the problems with EQA result interpretation, as there is a lack of consensus about how to determine group comparison data (8). In addition to harmonization, there needs to be an understanding about the utility and/or interpretation of the test and not just assess a median with a potential broad range of what would be graded or considered to be “acceptable” results. One such example would be a heparin EQA module, which typically includes PT, APTT, thrombin time and anti-Xa results. These tests are used to monitor unfractionated (UFH) or low molecular weight heparin (LMWH), and if necessary, dose adjustments are made based on reported result and target or therapeutic levels. A wide range of “acceptable” results (typically reported as the median and range) does not necessarily translate to clinical practice acceptability. Again, using the heparin EQA module, the LMWH anti-Xa test (heparin levels) with historically high CVs, the median may be 0.75 U/mL for LMWH, with an “acceptable” range due to reported variability of 0.3–1.5 U/mL, with a typical therapeutic target for LMWH being 0.5–1.0 U/mL. So, while the laboratories may be deemed to be “acceptable” using statistical criteria, those laboratories that reported results that may have resulted in dosing changes [less than 0.5 U/mL (requiring increased dose), or greater than 1.0 U/mL (requiring decreased dose)] and may have led to subsequent bleeding or thrombotic risks to patients. Therefore, a significant limitation to EQA programs is that they are not currently designed to adequately address whether the reported result would have impacted patient management, especially for tests that assess drug pharmacokinetics or pharmacodynamics.

EQA: other limitations

While most EQA programs do address a fair number of routine and esoteric hemostasis tests, the quality of these EQA samples are limited for a number of hemostasis tests. As such, without specific patient equivalent EQA samples, or lack of an EQA program to provide normal and abnormal samples for esoteric hemostasis assays, it is unclear how one can identify laboratories that perform poorly for different analytes due to a multitude of limitations (see Table 2). In the US, CAP surveys for comprehensive coagulation (module CGE) testing provides laboratories with samples to assess unusual tests such as factor VIII inhibitor and euglobulin lysis time, but rarely are these EQA samples abnormal. As some esoteric tests do not have adequate QC material (e.g., non-factor VIII inhibitors, ELT, platelet aggregation, etc.) there is a degree of uncertainty about the longitudinal accuracy of those assays in the absence of abnormal EQA samples. Both changes in clinical interventions and advances in diagnostic testing have many laboratories adding newer coagulation tests such as chromogenic factor VIII and factor IX testing, ADAMTS-13 testing, and DOAC to their testing menu due to clinician demands. However, US EQA programs are either devoid of these methods, or fail to provide adequate samples that mimic clinical practice. With the addition of extended half-life replacement therapy and non-factor replacement therapy for both hemophilia A and B, it is critical to have an EQA program that provides adequate proficiency testing material to not only assure quality for diagnostic utility, but also for patient management. An EQA program must include these novel therapies as an option for true quality assurance, as published data have demonstrated reagent differences in measuring these drugs (9).

EQA: alternative strategies

Due to the aforementioned EQA limitations, laboratories performing hemostasis testing may choose to seek alternative programs with better patient-equivalent samples to assure quality hemostasis laboratory practice. As a reminder, laboratories that perform regulated assays (for CMS) and graded (for CAP) must enroll in an EQA program that has been approved by the appropriate regulatory agency for those assays, but all other assays may be assessed using alternative EQA programs.

Several alternative coagulation EQA programs are available in the US (e.g., Table 1), but most of these focus on routine coagulation assays. One alternative special coagulation EQA program available in the US is through the North American Specialized Coagulation Laboratory Association (NASCOLA). This organization’s mission in part is to provide high quality, patient-based, EQA products to specialty coagulation laboratories in the US and other countries in North America (10). To ensure this quality EQA program, NASCOLA partnered with ECAT to provide specialty coagulation EQA, with modules including thrombophilia testing, ADAMTS-13 testing, factor VIII and IX inhibitors, DOACs, and others (11,12). Where possible, the NASCOLA/ECAT EQA uses samples derived from patients (providing true abnormal specimens) and evaluates post-analytical variables as well, with a larger peer group for assays usually restricted to larger or reference laboratories (e.g., ADAMTS-13, DOACs).

What is critical for an EQA program is to determine the commutability of a contrived sample to mimic or reflect the expected results of a patient sample (13). For certain measurands, the NASCOLA/ECAT EQA uses patient derived samples. Therefore, for some tests, (e.g., factor VIII and IX inhibitors), the quality of the EQA material is very robust at clinically important decision levels (1.0–5.0 Bethesda units). Another example would be for ADAMTS-13 activity, as the provided samples encompass normal and clinically significant abnormal samples (<20% activity), to assure that laboratories are providing accuracy in their assessment for patients suspected of thrombotic thrombocytopenia purpura (TTP), hemolytic uremic syndrome (HUS) or other microangiopathic thrombotic anemias (MTA). EQA samples with novel direct oral anticoagulants (dabigatran, rivaroxaban and apixaban) are available through the NASCOLA/ECAT program (14), with the comparison groups 2–3× larger than the CAP dabigatran and rivaroxaban comparison groups. Future initiatives for NASCOLA EQA include collaborative efforts with pharmaceutical companies that manufacture hemophilia replacement therapy drugs to provide specific EQA (or method validation samples) as a hemophilia replacement therapy module to assure post treatment test accuracy.

There are two NASCOLA/ECAT EQA modules that incorporate some assessment and measure of post-analytical analysis. The first module consists of a sample provided with patient history, and laboratories decide which tests to perform based on patient history; based on their results, a diagnosis is then submitted. This module was started a few years ago, with the first sample being that from a patient with mild hemophilia B. Secondly, there is an interpretive program for platelet aggregation studies (15). Typically, four platelet aggregation tracings from real patient evaluations are provided, along with relevant clinical history and laboratory results. While highly informative, this module may have some limited utility due to difference in methodologies (e.g., light-transmittance aggregometry or whole blood aggregation) and reagent concentrations between the EQA participating laboratory and the submitting laboratory. However, this module does provide some assurances that platelet aggregation studies have consistent interpretations based on a panel of agonists, the rationale for interpretations, as well as providing a bank for abnormal tracings that could be used for educational or other regulatory requirements (competency assessment) in the future.

Aside from ECAT and NASCOLA, the RCPAQAP is also partnering with ECAT and NASCOLA to provide a more robust platelet function testing EQA (16,17). This international collaboration is much welcomed (and needed) in the realm of special coagulation EQA. Perhaps other EQA programs will develop partnership and embark in the US marketplace as the needs for esoteric and better EQA programs become available. Current barriers for international EQA in the US would include custom importing issues and the need for “local” sponsorship. Advantages to international partnerships, aside from potentially better EQA material, is the potential to harmonize EQA reporting and assessing leading to better clinical practice (8,18).

Special coagulation EQA: conclusions

Special coagulation laboratories using a single source EQA program that does not readily provide the desired longitudinal assessment of accuracy (providing normal and abnormal samples) should consider alternative programs, with the caveat they must maintain those EQA programs required by their regulatory agencies. This practice would assure the competency and accuracy of a laboratory reporting esoteric coagulation results.

Acknowledgments

Funding: None.

Footnote

Provenance and Peer Review: This article was commissioned by the editorial office, Annals of Blood for the series “External Quality Assurance”. The article has undergone external peer review.

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/aob. 2019.07.03). The series “External Quality Assurance” was commissioned by the editorial office without any funding or sponsorship. Mr. Gosselin reports personal fees from Expert testimony for rivaroxaban and dabigatran testing, personal fees from Siemens Healthcare Diagnostics, personal fees from Diagnostica Grifols, personal fees from Machaon Diagnostic Laboratory, outside the submitted work. RA Marlar and DM Adcock serve as board members of the North American Specialized Coagulation Laboratory Association (NASCOLA).

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. NCCLS. A Quality System Model for Health Care: Approved Guideline. Wayne, PA NCCLS document GP-26-A, 1999.
2. Preston FE, Lippi G, Favaloro EJ, et al. Quality Issues in laboratory haemostasis. Haemophilia 2010;16:93-9. [Crossref] [PubMed]
3. Cunningham MT, Brandt JT, Chandler WL, et al. Quality assurance in hemostasis: the perspective from the College of American Pathologists proficiency testing program. Semin Thromb Hemost 2007;33:250-8. [Crossref] [PubMed]
4. Olson JD, Preston FE, Nichols WL. External quality assurance in thrombosis and hemostasis: an international perspective. Semin Thromb Hemost 2007;33:220-5. [Crossref] [PubMed]
5. Lippi G, Favaloro EJ. Preanalytical Issues in Hemostasis and Thrombosis Testing. Methods Mol Biol 2017;1646:29-42. [Crossref] [PubMed]
6. Gresele PSubcommittee on Platelet Physiology of the International Society on Thrombosis and Hemostasis. Diagnosis of inherited platelet function disorders: guidance from the SSC of the ISTH. J Thromb Haemost 2015;13:314-22. [Crossref] [PubMed]
7. Favaloro EJ, Lippi G. Post-analytical Issues in Hemostasis and Thrombosis Testing. Methods Mol Biol 2017;1646:545-59. [Crossref] [PubMed]
8. Favaloro EJ, Jennings I, Olson J, et al. Towards harmonization of external quality assessment/proficiency testing in hemostasis. Clin Chem Lab Med 2018;57:115-26. [PubMed]
9. Kitchen S, Tiefenbacher S, Gosselin R. Factor Activity Assays for Monitoring Extended Half-Life FVIII and Factor IX Replacement Therapies. Semin Thromb Hemost 2017;43:331-7. [Crossref] [PubMed]
10. North American Specialized Coagulation Laboratory Association (NASCOLA). Available online: https://nascola.com
11. North American Specialized Coagulation Laboratory Association (NASCOLA) PT/EQA. Available online: https://nascola.com/AccessibleServices/Testing
12. Peerschke EIB, Castellone DD, Ledford-Kraemer M, et al. Laboratory Assessment of Factor VIII Inhibitor Titer: The North American Specialized Coagulation Laboratory Association (NASCOLA) Experience. Am J Clin Pathol 2009;131:552-8. [Crossref] [PubMed]
13. Jennings I, Kitchen D, Kitchen S, et al. The importance of commutability in material used for quality control purposes. Int J Lab Hematol 2019;41:39-45. [Crossref] [PubMed]
14. Van Cott EM, Smock KJ, Chen D, et al. Testing for Dabigatran and Rivaroxaban by Clinical laboratories. Am J Hematol 2016;91:E464-E467. [Crossref] [PubMed]
15. Hayward CPM, Moffat KA, Plumhoff E, et al. External Quality Assessment of Platelet Disorder Investigations: Results of International Surveys on Diagnostic Tests for Dense Granule Deficiency and Platelet Aggregometry Interpretation. Semin Thromb Hemost 2012;38:622-31. [Crossref] [PubMed]
16. Favaloro EJ, Bonar R. External quality assessment/proficiency testing and internal quality control for the PFA-100 and PFA-200: an update. Semin Thromb Hemost 2014;40:239-53. [Crossref] [PubMed]
17. Favaloro EJ. Novel approaches to quality control and external quality assessment for platelet function testing with a focus on the platelet function analyser (PFA-100 and PFA-200). Ann Blood 2019;4:3. [Crossref]
18. Favaloro EJ, Gosselin R, Olson J, et al. Recent initiatives in harmonization of hemostasis practice. Clin Chem Lab Med 2018;56:1608-19. [Crossref] [PubMed]