In the article in this issue, “Quality monitoring approach for optimizing antinuclear antibody screening cutoffs and testing work flows,” Tacker and Perrotta address an issue faced by many laboratories—how to offer antinuclear antibody (ANA)3 testing that combines optimal clinical performance with realistic laboratory work flows (1). Specifically, Tacker and Perrotta evaluated an ANA enzyme immunoassay (EIA) as a way to decrease the number of manual immunofluorescence assays (IFAs) performed in their laboratory.
Maintaining IFA testing is a significant challenge for many clinical laboratories (2, 3). It is usually a manual process with many steps that each have risk for error. In addition, the interpretation is subjective, relying on well-trained clinical technologists for accurate and consistent test results. For years, clinical laboratories have recognized the need for alternatives to IFA that are more automated and less subjective. These alternatives are available in the form of EIAs and multiplex immunoassays (MIAs). EIAs generally use a human epithelial type 2 (HEp-2) cell lysate as the capture antigen, which makes it fairly representative of the IFA method (4). The MIA is a bead-based method in which individual antigens are conjugated to different bead sets (5, 6). Collectively, these beads represent an ANA by including the most clinically relevant antigen specificities. Despite the availability of these assays, many of which are cleared by the Food and Drug Administration, laboratories are sometimes hesitant to replace the ANA by IFA (7). This can be attributed, at least in part, to a position statement published in 2015 by the American College of Rheumatology (ACR) (8). In this statement, the ACR “supports the immunofluorescence ANA test using HEp-2 substrate as the gold standard for ANA testing.” The position statement also highlights that laboratories using other solid-phase assays “must provide data to ordering healthcare providers on request that the alternative assay has the same or improved sensitivity compared to” the ANA by IFA. Because of this statement, many rheumatologists have been hesitant to accept any methods other than IFA for their ANA testing.
Tacker and Perrotta have taken a different approach—rather than completely replacing the ANA by IFA with another method, they sought to incorporate an algorithm that would decrease the number of samples tested by IFA. Specifically, they sought to “focus manual IFA testing on specimens more likely to be positive.” To achieve this goal, the authors incorporated a high-sensitivity EIA, with positive samples reflexing to a confirmatory IFA. To develop and validate this algorithm, Tacker and Perrotta used a 4-phase approach, which included preimplementation, verification, implementation, and postimplementation stages.
During the preimplementation phase, the authors' analysis of the clinical performance of their ANA-only testing revealed a specificity of 77%, i.e., a high percentage of false-positive ANA results. In fact, the number of positive ANA results for those without any connective tissue disease (CTD) was almost 3-fold higher than those with a CTD diagnosis. This frequency is consistent with previous reports showing that 25%–30% of normal individuals have a positive ANA result at a 1:40 titer (3). The choice of dilution for the IFA ANA screen is not defined by the ACR and therefore defaults to the laboratory to implement appropriate screening dilutions for their own assay (9–11).
Moving into the verification phase, the authors evaluated the diagnostic sensitivity and specificity of the EIA using a cohort of disease and nondiseased controls. This evaluation confirmed that the EIA method had a very high sensitivity, which is obviously a requirement of any test used for screening, at 100%. However, as expected, that sensitivity came at the price of lower specificity at 70%, even compared to the IFA. These data are essential because they demonstrated that the EIA had the required sensitivity to serve as the screening assay in their proposed diagnostic algorithm.
Next, Tacker and Perrotta entered the implementation and postimplementation phases. In these phases, the authors optimized the screening cutoffs at each step of the ANA testing algorithm for sensitivity and specificity. This step was achieved by integrating laboratory data with CTD diagnostic information retrieved for the implementation and postimplementation data sets. It is important to note that the necessity to improve work flow efficiency could not come at the expense of lower diagnostic sensitivity. Furthermore, adding an extra screening step before IFA analysis might simply add unnecessary testing and expense if a low cutoff was used. Titration of the assay cutoffs would help the laboratory maximize the new ANA testing algorithm and reduce unnecessary ANA titers.
During the implementation phase, the authors first screened ANA orders by using the automated EIA platform with the manufacturer's cutoffs followed by IFA analysis on all positives by using a 1:40 dilution. Analysis of sequential samples during this phase demonstrated that 64% of samples tested by EIA were positive and were reflexed to the IFA. In addition, the performance of the EIA screen retained its high sensitivity at 98%, although specificity continued to be low at 51%. This low specificity (and high false-positive rate of 40%) indicated that a positive result by the EIA only correlated with a CTD diagnosis a small percentage of the time. These data suggest there was an opportunity to raise the EIA screening cutoff. The authors were concerned not only with the number of samples that reflexed to the IFA, but also with the low number of confirmed positive results. Armed with the data from the implementation phase, the authors evaluated the impact of changing the cutoff of the EIA screen and the IFA confirmation. By use of ROC analysis, and through consultation with their rheumatology colleagues, the decision was made to increase the EIA cutoff at which samples would reflex and to increase the screening IFA dilution. After these changes were made, the authors found that they were now only reflexing 36% of samples screened by the EIA. Importantly, based on diagnostic chart review, the authors demonstrated that, although the cutoff for reflexing was increased, the sensitivity of the EIA remained high at 98%, but with a dramatic improvement in specificity to 78%. The diagnostic utility of the IFA improved as well, with increases in both sensitivity and specificity.
The second goal of the algorithm was to optimize the work flow for the IFA testing. In the postimplementation phase, Tacker and Perrotta demonstrated improvements in 3 key metrics. The number of IFA reflexed to titer decreased to 36%. Second, the number of slides consumed per day was reduced from 4 to 2, which contributed to a 2-h decrease in the total IFA processing time each day and a 1-h decrease in hands-on time. This result correlated with a 25% reduction in full-time equivalent required each day for ANA IFA testing. Compounded over time, especially with increasing test volumes, these savings will have a positive impact on laboratory efficiencies.
Through their ANA algorithm, Tacker and Perrotta demonstrate how data collected throughout various test phases—including development, verification, and implementation—can be used to consistently improve both efficiency and test performance. The study comes at a time when manufacturers are also seeking to reduce the burden of IFA testing through automation (12, 13). Several companies have recently introduced fully automated IFA platforms that provide comprehensive testing from liquid handling to digital pattern analysis. It is important to note that the specifics of the study by Tacker and Perrotta must be properly validated and cannot be directly transferred to other laboratories. Differences in reagents, manufacturers, microscope settings, and testing personnel all contribute to interlaboratory variability of ANA IFA testing. Additionally, pretest probability influenced by physician or institutional test ordering criteria can greatly affect diagnostic specificity for a given method. However, their approach is certainly something to be emulated. The use of patient test results for monitoring test performance and providing opportunities for quality improvement is a valuable tool in all areas of the clinical laboratory.
↵3 Nonstandard abbreviations:
- antinuclear antibody
- enzyme immunoassay
- immunofluorescence assay
- multiplex immunoassay
- human epithelial type-2
- American College of Rheumatology
- connective tissue disease.
see article on page 678
Authors' Disclosures or Potential Conflicts of Interest: No authors declared any potential conflicts of interest.
- Received February 15, 2017.
- Accepted February 21, 2017.
- © 2017 American Association for Clinical Chemistry