Abstract
Background: Clinical laboratories have focused on quality for more than 60 years. While analytic quality is considered excellent in most laboratories, nonanalytic quality is an area for focused improvement. One of our quality metrics, lost samples, has been tracked continuously for 25 years and has demonstrated steady improvement. Nonanalytic processes have become highly automated within our organization, which, we believe, was a major factor in reducing lost samples. We have also implemented numerous behavioral controls and completed many process reengineering projects that have had a demonstrable effect on lost sample rates. Our objective in this study was to determine the overall contributions of our error-proofing methods to reducing lost samples.
Methods: Using data spanning 25 years, we plotted the correlation between lost samples and the implementation dates for 8 major phases of automation along with 16 process improvements and engineering controls.
Results: The lost sample rate decreased nearly 100-fold. In Six Sigma terms, the 12-month moving average for lost samples currently hovers around 5.85-sigma, with several months at or better than 6-sigma. While implementation of process improvements, engineering controls, and automation all contributed to the reduction, automation was the most significant contributor.
Conclusions: The custom automation in use by our laboratory has led to improved nonanalytic quality. Although this level of automation might not be possible for all laboratories, our description of 16 behavior and engineering controls may be useful to other laboratories seeking to design high-quality nonanalytic processes.
Impact Statement
Nonanalytic errors now exceed analytic errors as detractors from overall quality in clinical laboratories. Automation and process improvement/engineering controls have the potential for significantly improving a laboratory's nonanalytic quality. This report describes how advanced automation, combined with process improvements, enabled our laboratory to achieve Six Sigma quality levels in one nonanalytic metric, lost samples, improving the quality and reliability of the critical services we provide to patients. The report outlines how other clinical laboratories can use automation and the 16 described process improvement/engineering controls to significantly improve their own nonanalytic quality.
In response to growing concerns about the accuracy and reliability of laboratory testing, the College of American Pathologists Inspection and Accreditation program, established in 1961 (1), focused on analytic quality. In the ensuing decades, practices such as daily quality control, proficiency testing, and statistical assessment of quality were institutionalized worldwide to address analytic imprecision and bias. The majority of quality issues are now nonanalytic (2–5). Nonanalytic error is defined as unintentional, nonconforming events during the pre- and postanalytic stages. One type of nonanalytic error, samples lost before or after testing, is the focus of this report.
Materials and Methods
In this report, we express our results in terms of Six Sigma performance levels. Six Sigma performance is understood as ≤3.4 defects per million opportunities (DPMO)5 or 99.99966% defect-free work. Six Sigma performance is often described as world-class performance. Throughout this work, the Westgard Six Sigma Calculator (6) was used to convert DPMO to Sigma metrics.
Our approaches to lost samples fell into 2 improvement categories: (i) eliminating error potential through automation and (ii) managing human behaviors through process improvements and engineering controls. The following chronology lists the 8 automation stages and 16 process improvements/engineering controls we used. This list also serves as a key to Figs. 1 and 2.
Sigma metrics were calculated (6) using total lost samples/million billed units enterprise-wide. Billed units more accurately estimate DPMO opportunities. However, the change in Medicare's definition of a billed unit prevented using billed units before 1997.
Sigma metrics were calculated (6) using only samples lost within the areas served by the automation per million reference lab samples received. Samples were used instead of billed units in order to show the improvements before 1997.
Automation
1998. An automated transport and sorting system, built by MDS AutoLab, was installed (7).
2003. A 2-story freezer (−20 °C) automated storage and retrieval system that could hold up to 2.35 million postanalytic samples in stainless steel trays was designed and installed by Daifuku America. Specimen tubes requested by employees were retrieved by an automated sorter (Motoman Robotics Division, Yaskawa America).
2004. The AutoLab track system was significantly expanded, increasing hourly throughput from 2000 to 5000.
2004. Two Motoman storage autosorters, with capacities of 1000 samples per hour, were connected to the AutoLab track system to automatically place samples into storage trays.
2006. Four additional sorters were added to the AutoLab track, making 8 total and increasing total sort groups from 90 to 240.
2009–2010. A “Sort-to-Light” system was designed, built, and installed by our in-house engineers to automate the sorting of high-risk samples that could not be transported by the main automation system (i.e., critical frozen samples and samples in containers not meeting the dimensional requirements of the track). Comprising 15%–18% of the total volume, these were previously sorted manually, leading to a higher potential for loss. (A video demonstrating this system is at http://www.aruplab.com/testing/automation/videos.)
2010. A custom, high-throughput storage autosorter was built and installed by Automated Tooling Systems (ATS), sorting up to 4000 postanalytic samples per hour into storage trays.
2014. A new track system that used the MagneMover LITE® track (from MagneMotion), 10 robotic sorters plus 7 other robotic machines (built in-house), and 3 automated thawing and mixing workcells (built by Motoman) was installed. This system significantly increased automated transport and sorting capacity, bringing the total number of sort groups to more than 320.
Process improvements and engineering controls
1992. A formal procedure with customized checklists was implemented for missing sample searches.
1997. A standardized, false bottom, screw-cap transport tube (Sarstedt #62.612.016) was implemented to increase the percentage of samples handled by the automation system.
1997. Single-piece sample processing was implemented to replace an assembly line system, eliminating processing handoffs and lost sample potential.
1998. Raised edges were installed around sample-processing workstations to provide a barrier to control rolling tubes.
1999. A programming change was implemented for the automated storage system to reject samples submitted for storage that still had pending orders.
1999. Trash and biohazard waste receptacles were moved from underneath sample-processing workstations and were fitted with rounded covers that incorporated narrow, diagonal openings to eliminate the potential for unintentional sample discard.
2000. Skirting material was installed around the base of all equipment in sample-processing areas.
2003. Light fixtures were aligned with sample-processing workstations, eliminating shadowed areas.
2005. The lost sample checklists were updated to include a periodic “review and revise” requirement.
2009. Sample-processing staff implemented daily scheduled visual sweeps by assigned staff with documented inspection of floors, under workstations, and behind equipment.
2010. The lost sample checklist procedure was expanded to include a pattern analysis of where misplaced samples were either found or the last documented “touch” occurred.
2010. Sample-processing staff was organized into 4-person pods (teams) for processing shipments. Each pod had immediate access to dedicated processing experts, quality staff, and client support specialists.
2011. A paraffin tissue and extracted nucleic acid transport submission kit was designed and distributed to clients, ensuring that any combination of paraffin blocks, associated slides, and DNA scrolls stayed together during the transport and processing steps. A clear plastic sleeve enabled processing staff to view the contents without removing them.
2011. Batch receipt of incoming shipments from freight delivery services was replaced by barcode scans of each individual container at the time of receipt.
2012. The use of big data was implemented to build reports to alert staff to the potential for missing samples, containers, and shipments.
2014. A clean line of sight across and under sample-processing areas was established.
Results
The results of our 25-year experience are demonstrated in Figs. 1 and 2. The figure legends explain the data sources, the differences in the respective denominators used for each figure (billed units vs received samples), and the reason for Fig. 2 starting in 1991, vs 1997 for Fig. 1. The 2 figures together accurately depict the contributions of the 2 approaches to our error-proofing efforts. In both figures, sustained improvement after each automation stage (A through H) is demonstrated.
The configuration of our operations gave us the opportunity to test our premise that the automation stages had a greater impact than the process improvement/engineering controls. Our enterprise includes a reference laboratory in a business park and a separate hospital laboratory. The automation described above has only been implemented at the reference site. Since 2012, data have been collected according to the last “touch”—the last documented location of the sample. This step allowed us to compare metrics between the hospital complex and the reference laboratory. The data, using a paired 2-sample t-test for means, showed that the performance means for the 2 data sets were not the same (t47 = −5.238; P ≤0.05). For the first data set (the enterprise setting), the mean was 19.84 DPMO (SD 18.46); for the second set (the reference site), the mean was 6.19 DPMO (SD 2.672). The engineering and process improvements have been shared across all processing sites. Thus, the magnitude of the difference in DPMO values indicates that automation was responsible.
Discussion
Over the 25 years of this study, data collection practices remained consistent, despite a 30-fold growth in monthly samples over this time. Because the 8 automation stages were interwoven with the 16 nonautomation projects, it was difficult to assess the impact of any single intervention. We believe the overall influence of these activities is compounding, each building on the success of the last. Our results demonstrate that 2 approaches—automation and behavioral controls—working together, yielded significant results and suggest that reaching the 6-sigma target would not have been possible without automation.
Enhanced workplace conditions were an added benefit of the interventions noted here. For instance, transitioning to single-piece flow reduced handoffs and improved throughput efficiency. Improved lighting reduced labeling errors. Standardized transport tubes reduced inventory. Organizing and standardizing work areas improved workplace safety. Many of the big data reports were used to improve other nonanalytic processes. Finally, the automated systems we have described improved efficiency, effectiveness, turn-around time, and safety.
In 2005, Howanitz (8) reviewed reported error rates in clinical laboratories, confirming a marked difference between analytic and nonanalytic error rates. These findings paralleled other reports (2–5) that nonanalytic errors exceeded analytic errors in overall laboratory quality. Predictably, in recent decades, increased attention was directed to reducing the error potential for nonanalytic processes.
Nonanalytic processes are largely carried out using a series of manually executed steps that rely on human attention and memory. Yet, using behavioral interventions alone to effect exceptional quality may be beyond what is possible for many laboratory processes. Standard process redesign tools and engineered controls enable a higher level of quality, but processes that rely on human effectors will exhibit higher degrees of variability and lower levels of reliability. Astion (9), basing his reasoning on concepts originally presented by Resar (10), noted that eliminating human error requires automation, robotics, software, and advanced process design.
This report provides impetus and encouragement for other laboratories seeking to improve nonanalytic quality. Although custom automation was a major contributor to our improved lost sample metric, the value of available automation and behavioral/engineering interventions cannot be dismissed. We recognize the importance of a whole-organization, multidimensional approach to solving complex problems. Our experience advocates for relentless, iterative effort using a wide range of techniques to achieve Six Sigma levels of quality and ensure patient safety.
Additional Content on this Topic
Nonanalytic Laboratory Automation: A Quarter Century of Progress
Charles D. Hawker. Clin Chem 2017;63:1074–82
Acknowledgments
We thank Jonathan Genzen, MD, PhD, and Ronald Weiss, MD, MBA, for their reviews of the manuscript and helpful suggestions; Olivia Carril, PhD, for assistance with statistical analyses; and D'Arcy Grenz for assistance with the graphics in Figs. 1 and 2.
Footnotes
↵5 Nonstandard abbreviations:
- DPMO
- defects per million opportunities.
Authors' Disclosures or Potential Conflicts of Interest: Upon manuscript submission, all authors completed the author disclosure form.
Employment or Leadership: B.L. Messinger, C.D. Hawker, ARUP Laboratories.
Consultant or Advisory Role: None declared.
Stock Ownership: None declared. Honoraria: None declared.
Research Funding: None declared.
Expert Testimony: None declared.
Patents: None declared.
- Received January 23, 2017.
- Accepted March 20, 2017.
- © 2017 American Association for Clinical Chemistry
References
In this issue
