Development of e-Tool: In early 2012, the SLMTA team in Vietnam developed an electronic SLIPTA checklist tool. The e-Tool was pilot tested in Vietnam in mid-2012 and revised. It was used during SLMTA implementation in Vietnam and Cambodia in 2012 and 2013 and further revised based on auditors’ feedback about usability.

Outcomes: The SLIPTA e-Tool enabled rapid turn-around of audit results, reduced workload and language barriers and facilitated analysis of national results. Benefits of the e-Tool will be magnified with in-country scale-up of laboratory quality improvement efforts and potential expansion to other countries.

The SLMTA programme in Vietnam and Cambodia began in 2010 when representatives from each country participated in a two-week Training-of-Trainers workshop. After stakeholder engagement, governmental approval and, in the case of Vietnam, planning with international partners, baseline audits were conducted in seven laboratories in Cambodia in June of 2011 using the standard paper-based SLIPTA checklist. Auditors and implementing leadership from Vietnam noted that using the paper-based checklist created several challenges. Cumbersome audit and administrative processes meant that auditors were required to bring blank paper-based checklists and, in the case of exit or follow-up audits, previously-completed checklists to review prior recommendations and scores. Scores for all 111 questions had to be calculated and/or graphed manually for each report in each round of audits. Additionally, communication opportunities were lost; data analysis was inefficient and delayed; reports were not standardised; and there were difficulties in combining data for regional- or country-level management.

As a result of the observed difficulties, prior to performing the baseline audits in Vietnam, Ministry of Health (MOH) laboratorians supporting SLMTA in Vietnam set out to develop an electronic tool (e-Tool) for collection and use of audit data. This paper describes the development of the SLIPTA checklist e-Tool and discusses its benefits for SLMTA implementation.

The Vietnam SLMTA team pilot tested the e-Tool in early 2012 in parallel with the paper-based checklist in a provincial public health laboratory. The e-Tool was then used for the baseline audits of a mix of 12 public health, hospital and HIV laboratories in Vietnam in May 2012. The tool was further enhanced in an iterative process of feedback and improvement. Exit audits of the first round of SLMTA-supported laboratories in both Cambodia (January 2013) and Vietnam (June to July 2013) provided additional opportunities to refine the tool and validate macro formulae.

The e-Tool⁶ addresses several issues related to ease of use during the audit process and solutions for data management (Table 1). Drop-down response selections enable automatic scoring, easily accounting for non-applicable questions. Additional features of the electronic format include: automatic linkage of information; pre-programmed calculations to compare baseline and exit audits; and pre-set data check limits so as to help improve scoring accuracy. All comments entered in each of the 12 sections are automatically pulled to the end of the report to complete the recommendation section. Summary pages visually highlight the accomplishments and remaining gaps to be addressed, utilising clear colour-coded graphs.

In addition to improved functionality of data at the laboratory-level, the e-Tool improves usability for country-level programme management. The ability to merge data easily from multiple laboratories affords an accurate analysis with robust statistical indicators, including means and medians, minimum and maximum values, interquartile ranges, standard deviations of means and coefficients of variation at both indicator and section levels. Compiling results electronically for multiple laboratories allows the remaining gaps to be addressed by appropriate improvement projects in the SLMTA programme and shared with MOH leadership (Figure 1).

FIGURE 1: Example of e-Tool data output for baseline (row 1) and exit audit (other rows) for Section 1 (Documents and Records) compiled from 12 laboratories in Vietnam, 2013.

Data collected from all laboratories can be collated and used to assess countrywide gaps and initiate corrective actions. Figure 2 presents a sample e-Tool report, which combines results from all 12 participating laboratories in Vietnam. Baseline audits showed major gaps in four sections: internal audit (0%), corrective action (0%), occurrence management and process improvement (0%) and documents and records (22%) (Figure 2a). Section-level reports break out details of each question. For example, the Section 1 issues pertained predominantly to the development of manuals and Standard Operating Procedures (SOPs) (Figure 2b). Additional training was conducted on SOPs for the laboratory system, lectures on development of manuals and internal audit were included in the third SLMTA workshop and improvement projects related to these gaps were organised. Exit audit results showed substantial increases in the scores of these sections (100%, 57%, 87% and 88%, respectively) (Figure 2a), as well as in the subsection scores for Section 1 (Figure 2c). The clear visual summary of results provided by the e-Tool report facilitated timely development of an action plan to address the issues.

FIGURE 2: Visualised results for 12 laboratories in Vietnam using the SLIPTA e-Tool. (2a) Spider plot of combined baseline (May 2012) and exit (June to July 2013) audit median data for all 12 laboratory sites. (2b) Computer screen shot of baseline audit data for Section 1 (Documents and Records) of the SLIPTA checklist with bar-chart results. Colours indicate results that are acceptable (green), unacceptable (red) or partially compliant (yellow). (2c) Computer screen shot of exit audit data for Section 1 (Documents and Records) of the SLIPTA checklist with bar-chart results.

The ability to leave a one-page printed summary in Vietnamese for each laboratory was a substantial improvement over the paper tool. The communication benefits of providing same-day, on-site language-appropriate results to laboratorians and administrators encourage engagement from leadership in the improvement process. Furthermore, presenting reliable results in a graphical format leads to better understanding of problems and use of data for improvement.

As with the TB-LAT e-Tool, this first version of the SLIPTA e-Tool has capacity for data exporting; however, some minor issues have been identified when transferring data between various versions of Microsoft^® Excel. Improvements to the tool should continue, based on feedback from end-users. In addition, studies are needed to provide a formal evaluation of the tool, including time and cost savings, improved accuracy and user preferences.

Potential benefits of the e-Tool expand beyond improved auditing. The tool incorporates auditing instructions for each SLIPTA question, which will enhance learning and consistency for laboratories performing internal audits. Furthermore, as additional auditors are trained, the e-Tool can be used for evaluation and validation of their competency and standardisation of audit criteria. Moreover, when unlocked, the electronic checklist can be customised and expanded to address additional laboratory- or country- specific data needs.

The SLIPTA checklist e-Tool is now available for use by other countries and can easily be customised as needed; for example, optional languages can be added for local settings. Widespread use of the tool will allow development of a database for SLMTA programme managers at the global level in order to evaluate systematic gaps in laboratory quality and to enhance overall planning, implementation and sharing of results.

This project has been supported by the US President’s Emergency Plan for AIDS Relief (PEPFAR) through the CDC under the terms of Cooperative Agreement project number PS001928 (Vietnam Administration for Medical Services) and PS001285 (American Society for Clinical Pathology).

2.Yao K, Maruta T, Luman ET, et al. The SLMTA programme: Transforming the laboratory landscape in developing countries. Afr J Lab Med. 2014;3(1), Art. #194, 8 pages. http://dx.doi.org/10.4102/ajlm.v3i1.194

3.Yao K, Luman ET, SLMTA Collaborating Authors. Evidence from 617 laboratories in 47 countries for SLMTA-driven improvement in quality management systems. Afr J Lab Med. 2014;3(2), Art. #262, 11 pages. http://dx.doi.org/10.4102/ajlm.v3i2.262

4.Clinical and Laboratory Standards Institute. Application of a quality management system model for laboratory services; Approved Guidelines – Third Edition. [CLSI document GP26-A3]. Clinical and Laboratory Standards Institute; 2004.

5.Integrated Quality Laboratory Services (IQLS). TB-LAT tuberculosis electronic laboratory assessment tool [document on the internet]. c2013 [cited 2014 Apr 28]. Available from: http://www.iqls.net/docs/TB-LAT_presentation.pdf

6.Thuong NT, Pierson A, Hoang QTT, et al. Vietnam SLMTA E-tool [document on the internet]. c2013 [cited 2014 Apr 28]. Available from: http://aslm.org/aslm2012/images/docs/Speaker_Presentations/Sunday_2_Dec/Symposium_Overview_Day1/2B_-_Vietnam_SLMTA_E-Tool.pdf

7.Carvalho HM. SLMTA in Vietnam – The roadmap for laboratory quality. Vietnam: US Centers for Disease Control and Prevention; 2012 (Unpublished).

8.Ministry of Health of Vietnam. Laboratory QMS in Vietnam – Roads towards improvement. Report presented at the Symposium on Quality Management for Medical Laboratories. Bangkok, Thailand; 2011 March 17–18.

9.Kingdom of Cambodia, Ministry of Health. Appendix 17: Complementary package of activities for referral hospitals 2006–2010. In: National guidelines on complementary package of activities for referral hospital development from 2006 to 2010. 2nd ed., 2006; p. 128–130.