|
|
| |||||||||||||||||||||||||
• Introduction • Research method and design • Implementation sites • SLMTA implementation • Evaluation of laboratory performance • Quality improvement projects • Mentorship and additional support • Results • Overall performance • Performance of quality system essentials • Quality improvement projects • Effect of changes to the audit checklist • Discussion • Limitations • Conclusion • Acknowledgements • Competing interests • Authors’ contributions • References |
|||||||||||||||||||||||||
Objective: To describe the impact of SLMTA and discuss factors affecting the results, with an emphasis on mentorship. Methods: The SLMTA programme was implemented from April 2010 through November 2012. Participants attended three workshops and executed quality improvement projects; laboratory auditors evaluated performance using a standard checklist. The virology laboratory did not receive mentorship; however, the tuberculosis laboratory had an international mentor who visited the laboratory four times during the programme, spending two to four weeks embedded within the laboratory during each visit. Results: There was an overall improvement in the performance of both laboratories, with the virology laboratory increasing 13% (from 80% at baseline to 93% at exit audit) and the tuberculosis laboratory increasing 29% (from 66% to 95%). These scores were maintained nine months later at the surveillance audit. Conclusion: The SLMTA programme resulted in improved and sustained quality management performance for both laboratories. Mentoring was a possible factor in the substantial improvement made by the tuberculosis laboratory and should be considered in order to augment the training received from the SLMTA workshops.
To strengthen the laboratory systems of African countries in a systematic approach, the US Centers for Disease Control and Prevention (CDC), in collaboration with the American Society for Clinical Pathology, the Clinton Health Access Initiative and the World Health Organization’s Regional Office for Africa (WHO AFRO), launched the Strengthening Laboratory Management Toward Accreditation (SLMTA) training programme in 2009.4 This programme, which focuses on strengthening laboratory management to achieve immediate laboratory improvement and accelerate accreditation preparedness, has been implemented in 47 countries worldwide and is expanding rapidly.5 The programme includes workshops, improvement projects, site visits and, in some cases, mentoring.6 Nigeria embraced the SLMTA programme in 2009 and, by 2010, seven laboratory experts were trained to roll out the SLMTA programme in 24 of the 344 Nigerian laboratories supported by the US President’s Emergency Plan for AIDS Relief (PEPFAR). The Nigerian Institute of Medical Research (NIMR) has two reference laboratories, namely, the Human Virology Laboratory (virology laboratory) and the National Tuberculosis Reference Laboratory (TB laboratory), under its mandate ‘to conduct basic, applied and operational research for the prevention and control of communicable and non-communicable diseases of public health importance in the country in collaboration with the federal and state ministries of health and other stakeholders’.27 Both laboratories were amongst the 24 enrolled in the SLMTA programme. The two laboratories are similar in their institutional management and were supported by the same PEPFAR implementation partner. However, the TB laboratory was assigned a mentor to assist with SLMTA implementation, whilst the virology laboratory was not. We describe the impact of the SLMTA programme and discuss potential factors affecting the results, with an emphasis on mentorship.
The TB laboratory was established to meet the institute’s research mandate. From 2005, the scope of services increased with inclusion of a ‘directly observed treatment short-course’ (DOTS) diagnostic centre, which brought about the expansion and renovation of the laboratory in order to meet the TB diagnostic service needs of both the private and public sectors. With the DOTS centre, many more TB suspects were referred to the laboratory for diagnosis, treatment and follow-up, and the diagnostic workload increased dramatically. The laboratory was commissioned as a National TB Reference Laboratory in 2007 and offers the following services: smear microscopy for acid-fast bacilli; solid and liquid culture; identification of Mycobacterium tuberculosis complex and Mycobacteria other than tuberculosis (MOTTs); and drug susceptibility testing using solid, liquid and molecular-based techniques. The laboratory is also involved in national TB drug resistance surveillance. A total of 80 799 tests were conducted in 2010. During the course of the SLMTA programme, the TB laboratory included 14 degree-holding staff, four diploma-holding staff, three microscopists, one administrative staff, one data clerk and one cleaner. Each laboratory also had a director, laboratory manager, quality assurance officer and dedicated personnel who had consistent job responsibilities throughout the duration of the programme. SLMTA implementation The SLMTA programme was implemented in the NIMR virology and TB laboratories over two years and seven months (Figure 1). Three workshops were conducted within this period, with an average eight-month interval between them. The laboratory managers and quality assurance officers from both laboratories attended the workshops, after which they trained the other laboratory staff.
Evaluation of laboratory performance A baseline audit was conducted in each laboratory in April 2010, six months before the first SLMTA workshop (Figure 1). Intermediate audits were conducted after each workshop in order to monitor progress and to help identify any remaining gaps. An exit audit was conducted in November 2012, four months after the third workshop and a surveillance audit was conducted in August 2013, nine months after the exit audit, in order to determine the ability of the laboratories to sustain the quality advances made. The baseline and first intermediate audits were conducted using the laboratory accreditation preparedness checklist developed in 2009 by WHO AFRO.8 The checklist had a total score of 250 points distributed into 12 sections corresponding to the 12 quality system essentials. In 2012, WHO AFRO revised the checklist by adding more details in the requirements for documents and records and management review, as well as modifying the sectional scores, with a new total of 258 points.9 This revised checklist was used in parallel with the older version for the second intermediate and surveillance audits to allow us to evaluate the effect of the revision; results presented are from the revised checklist. For the exit audit, only the revised checklist was used. Based on the audit scores, laboratories were assigned a zero- to five-star rating, whereby < 55% = zero stars, 55% − 64% = one star, 65% − 74% = two stars, 75% − 84% = three stars, 85% − 94% = four stars and 95% − 100% = five stars. Independent laboratory experts who had taken the SLMTA training-of-trainers course, which included one day of training on auditing, were engaged as auditors; the same team conducted the audits in both laboratories, although different auditors were scheduled for each round of audit. Quality improvement projects Quality improvement projects were selected by each laboratory after the first and second workshops based on laboratories’ specific needs within topics addressed at the workshops. The projects were implemented by all staff members and were monitored for effectiveness by their supervisors using the WHO AFRO checklist; reports were presented at the next workshop by the quality managers. The virology laboratory embarked on three quality improvement projects after the first workshop. The first was that staff were trained on the importance of monitoring the autoclave with emphasis being placed on effective sterilisation and proper waste segregation. They were then assessed daily by means of an in-house-developed checklist in order to improve competency regarding sterilisation and waste disposal. In addition, provision was made for stock level on inventory cards, expired reagents were disposed of and general improvements were made to the organisation of the store; and the storage media for documents were monitored monthly to ensure ease of retrieval of records, documents and policies. After the second workshop, another set of improvement projects was conducted: complaint types, root causes, corrective actions and effectiveness were monitored in order to improve customer satisfaction; specimens in and out of the laboratory were clocked for three months to measure and improve turnaround time; and the number and duration of items out of stock were monitored in order to reduce stock-outs of materials, kits and reagents. The officer responsible for storage monitored the stock-outs from the weekly requisition and issue records. For the TB laboratory, two quality improvement projects were implemented after the first workshop: monitoring inventory of reagents in order to improve turnaround time for acid-fast bacilli smear microscopy; and training staff on how to conduct internal audits. The laboratory conducted three improvement projects after the second workshop: monitoring media preparation and reviewing sputum-collection records in order to reduce the contamination rate of cultured samples; administering and analysing questionnaires from clients and effecting corrective actions in order to improve customer satisfaction; and improving documentation of inventory and establishing a requisition system for the stores. Mentorship and additional support Contrary to SLMTA’s implementation roadmap,5 time constraints on the laboratory experts who rolled out the SLMTA programme in Nigeria prevented follow-up site visits at the virology laboratory between the workshops, which would have assisted in linking the training curriculum with on-site activities.For the TB laboratory, an experienced international mentor from the American Society for Microbiology was assigned to work with the laboratory throughout the SLMTA process. Only TB laboratories were assigned mentors in this round of the SLMTA programme in Nigeria. The mentor had a postgraduate degree in quality management systems and a doctoral degree in microbiology with a TB specialty, had previously managed a laboratory that successfully attained international accreditation and had attended a SLMTA training-of-trainers workshop. A facility-based approach was adopted as the mentor visited the laboratory on four occasions for an average duration of three weeks at a time, allowing an in-depth understanding of the laboratory. The mentor provided daily assistance to the staff in the implementation of QMS, which included training in practical skills, assisting in improving quality of testing and giving assignments to be completed between visits. Nonconformities reported from each audit were addressed by the mentor at each visit and management review meetings were established to identify opportunities for improvement and to formulate action plans. The mentor had administrative support from institutional management and the Federal Ministry of Health, as well as technical and logistical support provided by CDC’s office in Nigeria. Within the time frame of the SLMTA programme, the AIDS Prevention Initiative in Nigeria organised a five-day training on accreditation preparedness, with emphasis on quality management systems. Staff from both of the laboratories participated alongside staff from other laboratories that the organisation supports.
(Figure 2). The virology laboratory moved from an overall score of 80% at baseline, representing three stars, to 93% at the exit audit, representing four stars. The TB laboratory improved steadily from 66% at baseline audit, representing two stars, to 95% at exit audit, representing five stars. Both laboratories maintained these gains at the nine-month surveillance audit.
Performance of quality system essentials Examining the 12 quality system essentials closely revealed specific areas of strength, weakness and improvement (Figure 3). The greatest improvements for the virology laboratory were in purchasing and inventory (from 67% to 90%) and in process control and internal and external quality assessment (from 74% to 94%) (Figure 3a). The virology laboratory achieved 100% scores in five quality system essentials (documents and records; client management and customer service; internal audit; corrective action; occurrence and/or incident management and process improvement); however no progress was made in organisation and personnel, which remained at 80% for the exit audit. The TB laboratory generally started with lower scores than the virology laboratory, leaving more room for improvement. It made substantial improvements in management review (from 42% to 100%); internal audit (from 50% to 100%) and occurrence/incident management and process improvement (from 50% to 100%) (Figure 3b). The TB laboratory also had very good performance at the exit audit in all the quality system essentials, obtaining 100% scores in six sections (documents and records; management review; client management and customer service; internal audit; corrective action; and occurrence/incident management and process improvement).
Quality improvement projects The impact of the quality improvement projects on the quality system essentials is shown in Table 1. For the virology laboratory, after the first workshop the projects on documents and records and on purchasing and inventory impacted positively at the first intermediate audit and progress was sustained through the exit audit. The project on organisation and personnel did not have a positive impact on the score for the corresponding quality essential, as there was a drop in the next audit score, which did not improve beyond baseline at the exit audit. Of the projects conducted after the second workshop, only the client management and customer service project corresponded to sustained performance, with audit scores remaining at 100%, as they were at the baseline audit. Performance in information management was maintained at 93% after the second intermediate audit, but dropped to 83% at the exit audit, whilst purchasing and inventory improved from 93% before implementation to 97% at the second audit, but dropped to 90% at the exit audit. At the surveillance audit, improvements were generally sustained, except for purchasing and inventory, which reverted to the baseline score of 67%. For the TB laboratory, after the first workshop there was an improvement in the performance of the information management section, which continued through to the exit audit, where it reached 94% (Table 1). The internal audit score dropped initially, but improved subsequently, attaining 100% at the exit audit. After the second workshop, there was a gradual improvement for process control and internal and external quality assessment to 91% at the exit audit; and maintenance of client management and customer service at 100%. Purchasing and inventory scores decreased initially following project implementation (from 77% to 70%), then increased to 87% at the exit audit. The surveillance audit showed sustained or improved performance in all areas.
Effect of changes to the audit checklist Parallel audit scores using the revised (2012) checklist were slightly lower than those using the original (2009) checklist (Table 2). Differences ranged from 0% to 7% and were smaller at the surveillance audit than at the second intermediate audit, where they resulted in a change of star category for both the virology and TB laboratories.
Other countries, such as Kenya and Botswana, have found similar results when implementing an accreditation-readiness programme, with mentored laboratories showing greater improvement than their non-mentored counterparts.10,11,12 Whilst conclusive evidence is lacking, as none of these programmes were designed as case-control studies (i.e., with mentors randomly assigned to laboratories), the combined anecdotal evidence strongly supports the benefit of such mentorship. Mentors who spend extended, well-structured periods in the laboratory working alongside the staff and helping participants to put quality improvement into practice through direct, daily coaching, can provide the needed support to fast-track laboratories toward quality improvement. The laboratories faced several challenges with regard to SLMTA implementation. Firstly, whilst the laboratory checklist was used to help identify and correct problems, an understanding of some of the requirements was often a challenge, especially in the virology laboratory where an experienced mentor was not available to assist. Similarly, the virology laboratory reported challenges in interpreting ISO 15189 standard requirements and auditor recommendations. Secondly, though many of the quality improvement projects were implemented successfully and increased performance of the quality system essentials, some of these advances were not sustained, especially in the virology laboratory. The purchasing and inventory section was affected worst as some records were not maintained. Sustainability is a common concern for any improvement programme; once the intense focus of implementation ceases, special efforts and continued supervision are required so as to ensure that old habits do not return. It is possible that the root causes of the deficiencies were not properly identified and addressed. Nevertheless, improvements in total scores were sustained, suggesting that quality improvement overall was maintained. Successful achievement of the four to five star levels reached by the two NIMR laboratories indicates a high level of laboratory functioning and gives credibility to the quality of the laboratory test results produced for improved healthcare services. The 22 other laboratories in Nigeria’s first SLMTA round had similarly impressive results, moving from an average baseline of 60% to 87% at exit audit; 16 of the laboratories achieved four to five stars.13 These successes inspired Nigeria to implement a second round of SLMTA in 2013 and to begin discussions regarding further national expansion of the programme. Because of the potential benefit of on-site mentorship, national experts are being trained in Nigeria to play this critical role. Limitations Our observations are subject to several limitations. The first of these is that mentorship was not assigned randomly. Whilst factors that we examined, such as management support, laboratory size and testing volume, were similar for the two laboratories, there may have been other unobserved factors that could account for some of the differences. For example, the laboratories chose different quality improvement projects to implement between workshops. A report by Maina et al. suggests that internal audits (which were implemented by the TB laboratory after the first workshop) may be a catalyst for improvements in other areas, as conducting self-review can identify areas that need improvement.14 Whilst the virology laboratory was already conducting internal audits before SLMTA, the TB laboratory started with a 50% score in this area and increased to 100%, potentially helping to explain their improvement in other areas. The second limitation is that the checklists used for the baseline and exit audits were not exactly the same, potentially introducing bias in the results. Comparison of the scores obtained by the two checklists used in parallel at the second intermediate and surveillance audits revealed that the revised checklist produced slightly lower results than the original checklist, suggesting that our overall improvement results are possibly conservative. The final limitation is that the auditors engaged in this study had only undertaken one day of training on auditing, which is not adequate to fully qualify them as auditors. Whilst the use of a checklist helps to standardise the auditing process, some variability may have been introduced because of inexperience. Conclusion The SLMTA programm was successful in improving the quality of the laboratory systems in these two laboratories, as evidenced by improved and sustained audit scores. The laboratory with expert on-site mentorship improved farther and steadier, achieving a score of five stars. Our results suggest that laboratories should consider using on-site mentorship in order to augment the impact of SLMTA in implementing quality improvement.
The findings and conclusions in this article are those of the authors and do not necessarily represent the views of the CDC. This research has been supported by PEPFAR through the CDC. Competing interests The authors declare that they have no financial or personal relationship(s) which may have inappropriately influenced them in writing this article. Authors’ contributions R.A.A. (Human Virology Laboratory) analysed the data and initiated and wrote the manuscript. C.C.O. (National Tuberculosis Reference Laboratory) provided the required information for the TB laboratory. N.N.N. (National Tuberculosis Reference Laboratory) contributed to the write-up review. E.O. (AIDS Prevention Initiative in Nigeria) coordinated the implementing partner support. S.B. (American Society for Microbiology) implemented the mentorship model. O.O. (CDC, Nigeria) provided technical support. E.T.L. (CDC, United States) assisted with data analysis and interpretation and revised the manuscript extensively. E.O.I. (National Tuberculosis Reference Laboratory) was responsible for the overall oversight with regard to the project implementation programme.
2.Petti CA, Polage CR, Quinn TC, et al. Laboratory medicine in Africa: A barrier to effective health care. Clin Infect Dis. 2006;42(3):377–382. http://dx.doi.org/10.1086/499363 3.Audu RA, Sylvester-Ikondu U, Onwuamah CK, et al. Experience of quality management system in a clinical laboratory in Nigeria. Afr J Lab Med. 2012;1(1), Art. #18, 5 pages. http://dx.doi. org/10.4102/ajlm.v1i1.18 4.World Health Organization [WHO representative’s office for Rwanda]. Press release: Kigali host the launch of a program to accelerate national laboratory service capacity building towards accreditation in the African region [document on the Internet]. c2008 [cited 2014 Jul 30]. Available from: www.who.int/hiv/amds/diagnostics/amds_kigali_pr_lab.pdf 5.Yao K, Maruta T, Luman ET, Nkengasong JN. The SLMTA programme: Transforming the laboratory landscape in developing countries. Afr J Lab Med. 2014;3(1). In press. 6.Yao K, McKinney B, Murphy A, et al. Improving quality management systems of laboratories in developing countries: An innovative training approach to accelerate laboratory accreditation. Am J Clin Pathol. 2010;134(3):401–409. http://dx.doi.org/10.1309/AJCPNBBL53FWUIQJ 7.Nigerian Institute of Medical Research. Development of a strategic plan (2011–2015) [document on the Internet]. c2011 [cited 2014 Jul 27]. Available from: https://nimr.gov.ng/_data/NIMR_STRATEGIC_PLAN.pdf 8.World Health Organization’s Regional Office for Africa. WHO Guide for the Stepwise Laboratory Improvement Process Towards Accreditation in the African Region (with checklist) [document on the Internet]. c2011 [cited 2014 Aug 10]. Available from: http://www.afro.who.int/en/clusters-a-programmes/hss/blood- safety-laboratories-a-health-technology/blt-highlights/3859-who-guide-for-the- stepwise-laboratory-improvement-process-towards-accreditation-in-the-african- region-with-checklist.html 9.World Health Organization’s Regional Office for Africa. World Health Organization releases guide for the Stepwise Laboratory Improvement Process Towards Accreditation (SLIPTA) in Africa [page on the Internet]. c2013 [cited 2014 Jul 30]. Available from: http://www.aslm.org/stay-informed/press-room/news-articles/world-health- organization-releases- guide-for-the-stepwise-laboratory-improvement- process-towards-accreditation-slipta-in-africa/ 10.Gachuki T, Sewe R, Mwangi J, et al. Attaining ISO 15189 accreditation through SLMTA: A journey by Kenya’s national HIV reference laboratory. Afr J Lab Med. In press. 11.Makokha E, Mwalili S, Basiye F, et al. Using institutional mentorship to roll out SLMTA in Kenya. Afr J Lab Med. In press. 12.Mokobela K, Moatshe M, Modukanele M. Using SLMTA and mentorship to accelerate laboratory improvement in Botswana. Afr J Lab Med. In press. 13.Yao K, Luman E, SLMTA collaborating authors. Evidence from 617 laboratories in 47 countries for SLMTA-driven improvement in quality management systems. Afr J Lab Med. In press. 14.Maina, RN, Mengo DM, Mohamud AD, et al. Progressing beyond SLMTA: Are internal audits the key drivers of accreditation? Afr J Lab Med. In press. |
