key: cord-0728556-j66m020e authors: Ridikas, D.; Pessoa Barradas, N.; Migliori, A.; Katukhov, A.; Bode, P. title: Improved performance of neutron activation analysis laboratories by feedback workshops following interlaboratory comparison rounds date: 2021-06-14 journal: Accredit Qual Assur DOI: 10.1007/s00769-021-01473-9 sha: 453e9827592ff1346bfe71903059a723c3b982da doc_id: 728556 cord_uid: j66m020e The International Atomic Energy Agency (IAEA) implemented an innovative project for assisting neutron activation analysis laboratories in improving the validity of their results by feedback workshops for discussion of results from participation in interlaboratory comparisons rounds in 2010. The participants learned during these meetings to identify the most probable sources of errors in their analytical procedures and how to implement corrective actions to prevent reoccurrence. The outcome of successive rounds between 2010 and 2018 is discussed and experiences during the feedback workshops are given. The quantitative evaluation of the results shows an overall improvement in satisfactory performance. Moreover, there is a clear indication that improvements are consolidated in most laboratories but also stimulate laboratories to develop to a higher level of excellence. Regional differences in performance are also analysed. The International Atomic Energy Agency (IAEA) supports its Member States to increase the utilization of their nuclear research reactors, and the IAEA Technical Cooperation Programme (TCP) plays a major role in transferring nuclear knowledge and technology to its Member States. In addition to scientific research and training, research reactors are often initiated for providing commercial services and products, such as radionuclides for medical and industrial applications. An overview of opportunities for utilization of research reactors is available [1] from which it can be derived, as well as from the IAEA Research Reactor Data Base [2] , that the neutrons from miniature, small and medium-size reactors are in many cases used for neutron activation analysis (NAA). Over the years, the IAEA has stimulated the orientation of NAA laboratories worldwide to relevant fields of application. Whereas the Member States may have identified markets for NAA laboratories, demonstration of valid analytical data and organizational quality of the work process are preconditions for consolidating and expanding the stakeholder community. For these reasons, laboratories and/or stakeholders may prefer that the facility's management system is accredited for compliance with the International Standard ISO/IEC17025 [3] , and several NAA laboratories already accomplished this. One of the requirements in the process towards accreditation is that the laboratory provides objective evidence of the validity of its measurement results by, amongst others, participation in proficiency testing (PT) schemes by interlaboratory comparison. Participation in an interlaboratory comparison round may reveal that some results are not satisfactory. Laboratories then face the problem of finding the source of such non-conformity and applying, if relevant, effective corrective actions. Obviously, PT providers cannot deliver a laboratory and technique-specific after-care on potential sources of error and approaches to mitigate them. The IAEA therefore has implemented since 2010 a new mechanism for supporting NAA laboratories by a three-tier approach to assist the laboratories in assessing their proficiency in analysing material of various composition and effectiveness of implemented quality assurance and quality control approaches by: The IAEA facilitated participation in the WEPAL 1 's international soil and plant exchange programmes [5] for the rounds 2010-2017. Soil is considered to be an 'easy' material for NAA, whereas analysis of plant matrices may be more difficult given the much lower induced activities and risk of contamination. The number of elements to be reported is restricted by WEPAL's reporting format, where not all elements are present. In the years 2010, 2012 and 2015 two such rounds were facilitated. In 2018, the IAEA NSIL in Seibersdorf [6], with support of the IAEA TCP, provided and distributed two samples in an interlaboratory comparison round for proficiency testing: a marine sediment and an animal tissue. These samples mimic the analytical challenges for laboratories previously posed by the above-mentioned soil and plant material, respectively. Both materials had established degree of homogeneity and well characterized mass fractions of the major, minor and trace elements. Participants submitted their results also to NSIL for further data evaluation. In this 2018 round, each result had to be accompanied by an estimate of its uncertainty expressed as one standard deviation. No restriction on the number of the reported elements was imposed, and therefore many uncertified elements were also reported. Laboratories with highly experienced NAA practitioners as well as laboratories with newcomers in the technique participated in each regional (Africa, Europe, Asia and the Pacific, North America and Latin America) groups. These laboratories use different types of research reactors (varying from subcritical facilities, miniature reactors to medium-size reactors), differently configured gamma-ray spectrometers and different methods of calibration and data processing. WEPAL provided the IAEA an EXCEL file with all results of the IAEA facilitated NAA laboratories including WEPAL's own routine evaluation based on z-scores [7] . For the 2018 interlaboratory comparison exercise, NSIL calculated z-scores, u-scores and density distribution functions for each element with an assigned value or with a consensus value on basis of participants' results [8] . Typically, NAA laboratories reported mass fractions up to 25 elements in each sample of the soil-type materials, and up to 20 elements in each sample of the biological type. Sometimes the rescaled sum of z -scores, R SZ , is used as a singular indicator if more than one measurand is involved in a proficiency test: PT with x i the test result for the given measurand, x pt is the associated assigned value, and σ pt is the standard deviation for proficiency assessment of the specific measurand [7] . The drawback of this approach is that z-scores of opposite sign cancel out, thus masking extreme |z| values. The IAEA evaluated the performance of each laboratory based on the fraction (as percentage) of the submitted data reported for which the absolute z-value, |z| < 3. This is in agreement with clause 9.9.4 of the ISO13528:2015 [8] in which it is stated that 'In proficiency testing schemes that involve a large number of measurands, a count or proportion of the numbers of action and warning signs can be used to evaluate performance'. The performance of some NAA laboratories in previous interlaboratory comparison exercises indicated that under routine, i.e. day-to-day experimental conditions, at least 90 % of all reported data would meet this criterion, i.e. max. 2-3 outliers per sample. As such, the 90 % fraction was used as a target indicator for excellent performance of the NAA laboratories. Satisfactory performance was assigned to laboratories with 70 %-90 % of their data with |z| < 3. It should be noted that the z-scores do not account for the laboratory's own uncertainty of measurement. Many laboratories prefer therefore the zeta-score instead, which would often result in a smaller number of outliers, especially since the combined uncertainty of measurement may be equal to or larger than the standard deviation of the consensus value. The laboratories with less than 70 % of their data with |z| < 3 were categorized as 'in development'. This performance indicator (fraction, as percentage of submitted data with |z| < 3) allowed for monitoring the development of the improvement of a laboratory in successive proficiency testing rounds. All performance indicators are for sake of comparison only and are not based on international conventions. The feedback workshops were held as quickly as possible (with the exception for the first one) after completion of the last round in the respective periods (see Table 1 ). Participating laboratories were encouraged to select their representative as the person(s) that actually carried out the analyses. This recommendation was well followed-up, which facilitated evaluation, stimulated discussions and resulted in individual, bottom-up action plans for improvement. All participants presented, using a proposed IAEA template, details of their analytical procedure applied in the interlaboratory comparison testing. These details included, e.g. the sample masses, dry mass assessment, calibration procedure, corrections for neutron flux gradients, internal quality control applied and status of quality assurance implementation. In addition, participants provided their own view on their performance and, if relevant and possible, their hypothesis on sources of error. The results were further discussed within the broad platform of participants, moderated by the IAEA International Expert and IAEA Technical Officer for these projects. One of the observations was that, especially in the rounds organized by WEPAL, mass fractions of elements such as Al and Cr measured by NAA are sometimes significantly higher than the median of all results in such interlaboratory comparison exercises. In a few cases, this has led to |z| ≥ 3. It is assumed that the corresponding consensus values are sometimes slightly underestimated, as the majority of the other participants use techniques requiring digestion of the test portion and may not reach complete dissolution of some elements [9, 10] . As such, the NAA results might be more realistic estimates of the true values of the total mass fractions of these elements than the consensus values. However, the difficulties with the complete dissolution of Cr also resulted in a relatively large standard deviation of the consensus value that compensated the bias of the NAA results with the consensus value in the z score. No corrections were therefore applied in the IAEA metrics. Participants discussed several best practices in the different types of calibration in NAA, such as the selection of calibrators, neutron flux monitors and irradiation and counting geometry. Approaches were exchanged for managing nuclear reaction interferences by epithermal and fast reactor neutrons-such as for the measurement of the elements Al and Mg upon the use of the relative method of calibration in NAA, gamma ray spectral interferences and related peak fitting, like in the measurement of Sr via 85 Sr. The related practical recommendations have been incorporated in the IAEA e-learning course on neutron activation analysis [11, 12] . The test portions used in NAA vary from tens of milligrams to about 200-300 mg. The participants questioned it if some deviating results, especially for trace elements at low mass fractions, might be caused by insufficient degree of homogeneity of the materials. Several laboratories, either at newcomer facilities or with inexperienced staff had difficulties with trouble shooting and implementing corrective actions. To this end, the feedback The lessons learned during these feedback workshops were used for drafting action plans for improvements to be implemented; in addition, they were again distributed as a reminder to the participants at each new round. The following main sources of error were identified in the feedback workshops: • Insufficient study of the associated documentation from the PT provider on the sample handling (such as estimation of the moisture fraction, and minimum sample mass to be analysed) and on the required dimensional units in the reporting. • Shortage of valid calibrators/control materials and use of expired calibrators, often due to a lack of financial resources. Such constraints forced some laboratories to use substantially smaller amounts than prescribed of, e.g. reference materials. • Absence of independent internal quality (validity) control such as by simultaneous processing of reference materials and blanks, and no own strict acceptance criteria if such control materials are processed. Almost all participants, upon awareness of deficiencies in their analytical procedure and laboratory organization, initiated implementation of corrective actions (see, e.g., [13, 14, 15] ). This effected in improvements in the results in the next rounds. It is assumed that the analysis done in the proficiency testing rounds is representative for the quality of other 'routine' analyses performed by these laboratories. Participants were reminded during the workshops that Member State laboratories can submit requests to the IAEA for procurement of calibrators and (certified) reference materials. After reaching a very high level of performance in 2012/2013, some decline in performance for soil samples by laboratory 1 can be noticed, although the performance is still in the 'excellent' range. The NAA data analysis and interpretation is in some facilities no fully automated and requires several human interactions and transposing actions. Some errors indicate indeed mistakes like interchange of data but it can also not be excluded that less experienced staff did the most recent analyses. Laboratory 4 joined the project in 2015. Inexperienced staff did the analyses in the first round. The effect of the 2015 workshop is clearly visible in the improvement in performance; by 2018 they almost reached their target performance for the soil analysis (90 % of results with, |z| < 3). However, still a few improvements are needed for analysing material with low trace element values. The current performance levels of laboratories 1-4 is nearly on par with that of several other NAA laboratories that reached a very high number of satisfactory results already at the start of this project (laboratories 5-7 in Fig. 2) . The trends in the performance of all NAA laboratories participating in this project is depicted in Fig. 3 for both matrices: the relatively easy soil-type samples and the more challenging biological samples with low trace element levels. NAA is a mature technique, with a straightforward measurement equation [16] , but in practice expertise in the technique and knowledge of all effects in quantification, which leads to avoidance of potential sources of error, comes from hands-on experience. Different to other analytical techniques, NAA is rarely a part of the analytical chemistry curriculum at technical schools or in academia. The IAEA therefore supports its Member States in this area via expert missions and fellowship training. However, the relatively frequent turnover of experienced staff in some laboratories hampers the training and transfer of knowledge to new staff, which may have consequences for analytical performance, especially if no quality management system has been implemented. These issues likely underpin laboratories categorized in Figs. 3-7 as 'In development'. Such laboratories often have more difficulties with the low trace element level samples-and thus low induced activities, with lower count rates and resulting poor counting statistics. Further, experience is lacking on corrections for interferences, blanks, measurement geometries and coincidence summing effects, to name a few. In addition, absence of quality management also sometimes contributes to gross errors indicated in the section 'Lessons learned'. The differences in performance of NAA laboratories in the various IAEA regions Latin America and the Caribbean, Europe, and Asia and the Pacific can be derived from Figs. 4, 5, 6. There are, unfortunately, insufficient results from NAA laboratories in the regions Africa and North America for a meaningful comparison. The comparably better performance by the NAA laboratories in Latin America and the Caribbean may to some extent be considered as the impact of many joint IAEA/ARCAL 2 projects in this region with emphasis on implementation of quality assurance and quality control, and on the relatively less frequent staff turnover in those NAA laboratories resulting in consolidation and transfer of expertise. The largest interlaboratory comparison among NAA laboratories was held in the year 2019 with the support of the TCP regional project 'Enhancing Utilization and Safety of the Research Reactors' [17] . Fifteen European laboratories operating activation analysis techniques registered for this proficiency test, making it the largest regional contribution out of the 46 NAA laboratories from were not yet levelled to those of the others in this region, so it is expected that the lessons learned from continuing participation will further improve the overall regional performance in Europe. An increasing number of laboratories in the Asia and the Pacific region joined the project since 2015, and some of them had a disappointing performance at their first round (Fig. 6) . For example, no laboratory qualified for the category 'Excellent' in analysing biological-type samples in 2018. The main reasons identified have already been mentioned in the above: relatively frequent staff turnover and insufficient implementation of quality assurance and quality control procedures. In some cases, and in all regions, laboratories decided to have newly arrived staff or students perform the analyses, some of them with very little experience. These laboratories reported benefitting from participating under such conditions, as it gave them insight on critical areas, where improved quality assurance procedures and/or quality control mechanisms need to be implemented, and on the need to invest further in training. To support training, the IAEA E-learning course on neutron activation analysis [11, 12] includes ample guidance not only on the fundamentals but also on the practice and quality assurance/quality control of the technique, and is therefore widely used and acknowledged. Continuation of participation will lead to an improvement in performance of the laboratories currently categorized as 'Satisfactory' and 'In development', as observed in other laboratories shown in Fig. 1 . Many laboratories favour a comparison of results on basis of zeta-scores, as this accounts for the uncertainty of measurement. The reporting of measurement uncertainty in the 2018 round allowed for a comparison of the performance classification on basis of z-score and zeta score as shown in Fig. 7 . Using the zeta score ( and, similarly as for the z-score, the criterion |zeta| < 3) resulted in a larger number of laboratories within the best performance category 'Excellent', while reducing the number of those in the weakest category. This IAEA initiative to facilitate laboratories' participation in proficiency testing schemes complemented by a new approach of feedback workshops, expert missions and procurement of indispensable calibrators and reference materials, resulted in a measurable increase in the analytical performance of most participating laboratories. Several laboratories demonstrated consolidation of their already satisfactory performance over time. A new metric has been introduced for assessing the performance of the participants, based on the fraction of all data reported with a performance |z| score < 3. Such a metric has not been published in reports of other interlaboratory comparison exercises; therefore, it is not possible to derive any conclusions from comparison to other analytical multielement techniques. The results from the interlaboratory comparison rounds confirmed that many laboratories operating NAA have reached an operational level at which they periodically report satisfactory to excellent results for the last 5 years. A few laboratories, representing ~10 % of all participants in each round, did not show much improvement from their first participation and continue their efforts for better performance taking advantage of the lessons learned during the feedback workshops. Several NAA laboratories, especially in developing countries, have insufficient resources of their own to cover the fees of interlaboratory comparison testing schemes. Therefore, it has been advised to consider bi-or multi-lateral exchange of samples for analysis as an alternative. In Asia, the Forum for Nuclear Cooperation in Asia (FNCA) has an activity in which interlaboratory comparison testing is organized amongst NAA laboratories in FNCA member states [18] . Similar activities may be initiated amongst regional research reactor networks. In 2020, the COVID-19 pandemic interrupted this activity. The IAEA intends to continue organizing proficiency testing exercises for NAA laboratories from 2021 onwards, combining it with contributions from laboratories operating nuclear-related analytical techniques for measurement of elements, such as based on X-ray fluorescence. As in the past, the PTs will be followed-up by planned feedback workshops for elaboration on results and further improvement of the measurement processes. TECDOC No. 1715 'Commercial Products and Services of Research Reactors ISO/IEC 17025 General requirements for the competence of testing and calibration laboratories ISO/IEC 17043 Conformity assessment -General requirements for proficiency testing, ISO/IEC 2010 Wageningen Evaluating programs for Analytical Laboratories, www. wepal. nl, last visited WEPAL Statistics Statistical methods for use in proficiency testing by interlaboratory comparison (ISO 13528L2015(Cor.2016-10) Comparison of relative INAA and k0-INAA using proficiency test materials at ITU TRIGA Mark II research reactor Eds) International Conference for Innovation in Biomedical Engineering and Life Sciences Neutron activation analysis: a primary method of measurement The world's largest proficiency testing exercise for Neutron Activation Analysis laboratories by interlaboratory comparison: Focus on Europe The authors thank all NAA participating laboratories for their participation and contributions to the IAEA follow workshops, based on which this publication has been prepared. The IAEA Technical Cooperation programme is acknowledged for being the main vehicle for implementation of technology transfer projects to the IAEA Member States. WEPAL is also acknowledged for their special services in this project. The authors declare that they have no conflicts of interest.