PDF hosted at the Radboud Repository of the Radboud University Nijmegen The following full text is a publisher's version. For additional information about this publication click this link. http://hdl.handle.net/2066/112344 Please be advised that this information was generated on 2021-04-06 and may be subject to change. http://hdl.handle.net/2066/112344 trends in analytical chemistry, vol. 9, no. Z,l990 58 References C. L. Forgy, Artificial Intelligence, 19 (1982) 17-37. F. Barachini, 8th International Workshop on Expert Systems & their Applications, Avignon General Conference Vol. 2, EC2, Gerfau, Paris, 1988, pp. 217-234. A. Gupta, 5th International Workshop on Expert Systems and their Applications, Avignon, EC2, Gerfau, Paris, 1985, pp. 26-57. M. R. Genesereth, Proceedings on the 3rd AAAZ Conference, Kaufman, Los Altos, CA, 1983, pp. 119-124. S. Russell, The Complete Guide to MRS, Report No. STAN- CS-851080, Stanford University, Stanford, June, 1985, 121 PP. 6 B. Silver, Studies in Computer Science and Artificial Intelli- gence, Vol. 1, North-Holland, Amsterdam, 1986. 7 J. Treur, Znformatie, 30 (12) (1988) 909-972. 8 R. Davis and B. G. Buchanan, in B. G. Buchanan and E. H. Shortliffe (Editors), Rule-Based Expert Systems, Addison Wesley, Massachussetts, 1984, pp. 507-530. 9 M. L. Wright, M. W. Green, G. Fiegl and P. F. Cross, An Ex- pert System for Real-Time Control, IEEE Software, IEEE Computer Society, March, 1986, pp. 16-24. M. De Winter, D. Grietens and Dr. M. Rijckaert are at the Chem- ical Engineering Department, KU Leuven, De Croylaan 46,303O Heverlee, Belgium. An expert system for the validation of high- performance liquid chromatographic methods L. M. C. Buydens and J. A. van Leeuwen Nijmegen, The Netherlands M. Mulholland Cambridge, U.K. B. G. M. Vandeginste and G. Kateman Nijmegen, The Netherlands A method validation expert system, developed as part of Es- prit project ESCA, is described. The proper validation of high-peeormance liquid chromatography methods is an is- sue of rapidly growing importance. This is due to the in- creasing demands of good laboratory practice. Since vali- dation involves a lot of statistical calculation and interpreta- tion with which most analysts are not familiar, it is often ne- glected in the method development process. An expert system that provides this knowledge and experience is therefore very useful. Introduction Method validation is gaining rapidly in importance due to the increasing demands of good laboratory practice. This is especially true in the area of phar- maceutical analysis, where regulatory bodies pose increasing demands on the analytical methods. The Esprit project ESCA aims to build demonstrator ex- pert systems for method development in high-per- formance liquid chromatography (HPLC) analysis of pharmaceutical compounds. The general ap- proach of the project is described in the paper on ESCA, elsewhere in this issue’. Such a system should include all necessary steps from the selection of initial HPLC conditions up to the validation of the method. This resulted in four expert systems, each having a specific task covering the whole development pro- cess. The expert system for the selection of initial conditions provides chromatographic conditions that yield acceptable retention times for all peaks in the chromatogram. When a chromatogram is obtained in this way it may be necessary to optimise the selec- tivity to obtain an optimal distribution of the peaks over the chromatogram. This is done by the expert system for the selectivity optimization. The expert system for the optimization of chromatographic and instrumental parameters aims to obtain an accepta- ble resolution and signal-to-noise ratio in the short- est possible time. The task of the method validation expert system is to validate the HPLC method that has been selected and optimised in the other expert systems of ESCA. All these domains have been de- scribed elsewhere’>*. This paper focuses on the ex- pert system for method validation. In the subsequent sections the scope and aproach in this expert system is described. Scope of the method validation expert system Method validation is a very broad concept. The full validation of a method comprises the testing of 0165-9936/90/$03.00. 0 Elsevier Science Publishers B .V. trends’ in analytical chemistry, vol. 9, no. 2,199O 59 different aspects of its performance such as accura- cy, precision, sensitivity, selectivity and limitations. The expert system described here concentrates on the precision testing of an HPLC method. Precision testing is the determination of the random error. Many methods will be applied under slightly differ- ent conditions from those of their development. The method may be used on other instruments, by other analysts or even in other laboratories. Therefore an estimation of the precision is a very important aspect in the method validation. According to Youden and Steiner3 one can distinguish a repeatability test and a reproducibility test. Repeatability testing involves testing the performance of a method when repeated under the same conditions, on the same instrument and by the same analyst. The reproducibility of a method is its precision under changing conditions and environments, e.g. on other instruments, by other analysts or in other laboratories. A reproduci- bility test hence involves an interlaboratory study which implies much effort and expense. The interla- boratory test of a method is doomed to failure when the method is not rugged for some slightly changing environmental and operational conditions that can be expected when the method is transferred to an- other site. Before organizing a large interlaboratory reproducibility test, the method can be intralabora- tory tested for its ruggedness with respect to ex- pected changes. The evaluation of the precision often involves the use of more or less complex statistics. Most analysts therefore consider it a tedious task to test the preci- sion of a developed method. An expert system that proposes the experiments that should be carried out, performs the appropriate statistical tests and inter- prets the results, and subsequently providing advice on the respecification of the method if the test fails, would be very useful. The expert system that has been developed in the ESCA project provides such advice on the repeatability and the ruggedness test- ing of HPLC methods435. In this article we focus on the ruggedness testing part of the expert system’. Ruggedness system The use of statistical experimental design has been proposed by Youden and Steiner3 to evaluate the ruggedness of a method for certain factors. More re- cently these designs have been applied in ractice for the ruggedness testing of HPLC methods ?9 - . The ruggedness expert system described here con- tains the necessary knowledge and experience for advising the user on the use and interpretation of ap- propriate statistics. The expert system basically con- sists of a separate module for each of the following four steps: l method description; l the selection of important factors; l the selection of the experimental design; l processing and interpreting the results. The method description module The user is asked to fully specify the method that will be tested. This includes information about sever- al aspects of the analytical procedure, all of which may influence the degree and extent of the testing procedure. This module contains knowledge on the normal range of the specifications. If the system ac- cepts the user’s method description, the user can safely rely on the result of a further consultation. Only when the method description is very incom- plete is some caution required. The conclusions may not be fully significant when too much information is missing. In all other cases the user can be confident about the results when the method description is ac- cepted by the system. The requested information is: l information about the analytical procedure of the HPLC method, such as sample preparation, col- umn, flow-rate, etc. l chromatographic results, such as retention times and minimal resolution between two peaks. l intended application area of the method. Differ- ent levels of validation are required, depending on whether the method is only used a few times or is applied for many samples in different laborato- ries. Also, before a method is submitted to regula- tory bodies, a thorough precision test should be carried out. l finally, knowledge about the availability of instru- ments is needed to build up the full validation pro- cedure. The method description module guides the user in providing all necessary information, without asking for superfluous or irrelevant data. It contains com- mon sense knowledge about chromatographic prac- tice and asks only information that applies to the spe- cific case. The consultation of this module is the basis for the further steps in the ruggedness system. The factor selection module In this step all factors that are likely to vary in the daily use of the method and that are suspected to in- fluence its performance are identified. Since HPLC is a rather complicated analytical technique, many parameters must be considered. The experienced analyst has however more or less strong ideas about the most important factors that are relevant to the behaviour of the system. For example, a method should be rugged for the temperature when it is clear 60 trends in analytical chemistry, vol. 9, no. 2; 1990 from the method description that no temperature control is provided. When the method is to be used over a long period of time and/or in other laborato- ries, then it is clear that the temperature of the envi- ronment will vary. When the method is not rugged for these variations this can cause alterations in its behaviour and wrong results may be obtained. The factor selection module covers the knowledge about the significant selection of these factors. This in- volves experience about the suspected variations of the selected factors in different environments”. Another feature of this module is that the user is allowed to add or delete factors that are chosen by the expert system. The levels of the selected factors can also be altered. This feature is provided to allow the user to introduce his/her knowledge about a spe- cific practical situation, For good laboratory practice these alterations are registered so that they can al- ways be traced afterwards. When the user agrees with the proposed set of factors, all information is stored and the system can continue. Module to select an experimental design The number of factors that may influence the method’s performance, when applied in practice, is very large. Even when only the most important fac- tors are selected a large number of experiments is still required. Experimental designs that allow as much information as possible to be extracted from a minimum number of experiments are necessary. When it can be assumed that there are no higher or- der interactions between the different factors, which is often a realistic assumption, then the fractional factorial designs are the most efficient for this pur- pose3. The designs that are implemented in the ex- pert system are: l full factorial designs; l half fractional factorial designs; l saturated factorial designs; and l reflected saturated factorial designs. The actual choice of a particular design depends on the number of factors and levels that are selected by the factor selection module. When the effects of only two or three factors are to be established at two levels then the full fractional designs are selected, otherwise the saturated fractional designs are pre- ferred. When the expert system has made a decision it presents the selected design together with the batch of experiments that are to be carried out. At this stage the user leaves the expert system to per- form the required experiments. Module for processing and interpretation of results When the experimental design has been selected an appropriate spreadsheet is created where the re- sults of the experiments can be entered. When all data have been collected the expert system can start processing the results. First the relevant chroma- tographic results from each experiment are calcu- lated. These include: concentrations, peak efficien- cy, resolutions etc. When these parameters are ob- tained the calculations can start to find the effects of the different factors”. The method is supposed to be repeatable before the ruggedness test is started. To check if this is valid over the whole experiment all standard errors are examined. Since the analyte con- centrations are the most important chromatographic results, the main effects of all factors on the calcu- lated concentrations are calculated first and com- pared with a prespecified tolerance level. If any fac- tor shows an unacceptably large effect on the con- centrations, the method fails on the ruggedness test. The main effects on all the other parameters, such as peak height and resolution, are then investigated in the same way. When the tolerance level for these pa- rameters is exceeded, the ruggedness test does not fail, but warnings are flagged to the user. If the rug- gedness test of a specified method fails, the user is advised to respecify the method and to repeat the ruggedness test. For obtaining meaningful advice on this issue, this expert system must be linked to the other expert systems that were developed within the ESCA project. The most straightforward is the inte- gration with the expert system for the optimization of instrumental and chromatographic parameters. This expert system can give advice on how to change the method in order to obtain improvements, e.g. in resolution. Integration of these two expert systems is proceeding. When the method passes the ruggedness test a ruggedness report is printed. This includes a set of system suitability criteria. These are the maximum and minimum values that are found for chroma- tographic parameters such as resolution and peak height. Whenever the method is applied, the analyst can then use these values to evaluate the chroma- tographic system regarding its suitability for the method. Implementation The method validation system is implemented in Goldworks, one of the selected expert system shells in the ESCA project. Since method validation is a very complex and broad area, of which only a part is implemented in the present system, it was necessary to build the expert system in a way that is easily ex- tendable. We chose a modular approach. The two main modules that constitute precision testing are the repeatability and the ruggedness system. These modules can be consulted separately. Each of these trend in analytical chemistry, vol. 9, no. 2,199O 61 / I h sd w di0mos3 rqeet repeat A A Repeatability Supervisor A V method factcf select diagnose description choice design Neeechess A A A A A Rhggedness I ?/ \1/ v V VV \/ Common Datastructure I I 1 Y explain ( Fig. 1. Implementation architecture of the precision testing expert system. * Screens > chromatograph questions > options I CRROMATOGRAPH Main questions and answers I Related answers Flow rate (ml/min) : 1.5 Number of solvents :2 PR : 2.5 Buffer cone (M) : Additives : Injection volume (ul) : 10 Temperature mode : CONTROLLED J I I Minimum solvent (%) : 25 Solvent 1 (%) : 75 Solvent 2 (%) : 25 Solvent 3 (%) : Solvent 4 (%) : Minimum additive (%) : Additive1 :o %( Additive2 :o %( Additive3 :o %( Additive4 :o %( i Additive5 :o %( 1 Temperature (deg. C) : 40 Fig. 2. Example of user interaction with the ruggedness expert system. 62 main modules are in turn built from different submo- dules, and each submodule can also be consulted in- dividually. A structure, called the supervisor, guides the user through the appropriate modules. It con- tains meta level or strategic knowledge on the next module to be consulted and therefore needs an over- view of the results and data that are used in all sepa- rate modules. All data and results of the modules are stored in common database, accessible by all mod- ules and the supervisor. The overall architecture is pictured in Fig. 1. This architecture allows the integration of mod- ules which contain very different types of knowl- edge. This is very useful since, as can already be seen from the ruggedness system modules, very different knowledge sources are necessary. Experiential as well as algorithmic knowledge sources must be com- bined in an integrated system to make it useful. This combination of modules containing different types of knowledge is a typical feature of second generation expert systems12. In the ESCA project the stand-alone expert sys- tems described elsewhere in this issue’ will be inte- grated. The architecture of the method validation expert system is flexible enough to allow an easy in- tegration with the other expert systems. Though user-friendliness was not the main con- cern in the project, attention was paid to developing an efficient and rather robust user interface. The questions are presented to the user in a window sys- tem. Explanations and additional help are provided where necessary. These can always be accessed by the user through special pop up windows. An exam- ple of the user interface is presented in Fig. 2. Conclusions The expert system for method precision testing, developed within the ESCA project, is a succesful stand-alone expert system. Validation of the system by means of 11 real test cases resulted in about 85% of success. Even for the cases where the conclusions were different from the real expert, the expert sys- tem choice was acceptable to the expert and in some cases even better than the real expert. A full descrip- tion of the validation and further evaluation of the system is published elsewhere13. The implementa- tion of the system allows future additions and inte- grations to be carried out with flexibility. Acknowledgement Part of this research is supported by the EEC as Esprit project P1570 Expert systems in chemical analysis (ESCA). trends in analytical chemistry, vol. 9, no. 2,’ 1990 References 1 J. A. van Leeuwen, L. M. C. Buydens, B. G. M. Vande- ginste and G. Kateman, Trends Anal. Chem., 9 (1990) 49. 2 D. Goulder. T. Blaffert, A. Blokland. L. M. C. Buvdens. A. 9 10 11 12 13 Chhabra, A. Cleland, N. Dunand, H. Hindriks, ‘G. Kate- man, J. A. van Leeuwen, D. L. Massart, M. Mulholland, G. Musch, P. Naish, A. Peeters, G. Postma, P. J. Schoenma- kers, M. DeSmet and G. B. M. Vandeginste, Chromatogra- phia, 26 (1988) 237-243. W. J. Youden and E. H. Steiner, Statistical manual of the AOAC, AOAC, Washington, DC, 1975. M. Mulholland, J. A. van Leeuwen and B. G. M. Vande- ginste, Anal. Chim. Acta, 223 (1989) 183-192. M. Mulholland, N. Dunand, A. Cleland, J. A. van Leeuwen and B. G. M. Vandeginste, J. Chromatogr., in press. M. Mulholland and H. Waterhouse, .I. Chromatogr., 395 (1987) 539-551. M. Mulholland and J. Waterhouse, Chromatographia, 25 (9) (1988) 769-774. M. Mulholland, P. J. Naish, D. R. Stout and J. Waterhouse, Chemometrics and Intelligent Laboratory Systems, 5 (1989) 262-270. A. Mulholland, Trends Anal. Chem., 7 (1988) 383. J. A. van Leeuwen, B. G. M. Vandeginste, G. Kateman, M. Mulholland and A. Cleland, submitted for publication. G. Box, W. Hunter and J. Hunter, Statistics for Experiments, an Introduction to Design, Data Analysis and Model Build- ing, Wiley, New York, 1978, pp. 291-453. B. Maitre, T. Laasri, F. Mondot, F. Charpillet and J. P. Ha- ton, Proceedings of the 9th International Workshop on Expert Systems and their Applications, Specialised Conference on Second Generation Expert Systems, Avignon, May 29-June 2,1989, EC2, Nauterre, 1989, pp. 237-251. M. Mulholland, J. A. van Leeuwen and L. M. C. Buydens, J. Chromatogr., in press. Drs. L. M. C. Buydens, .I. A. van Leeuwen and G. Kateman are at the Department of Analytical Chemistry, Catholic University of Nijmegen, Nijmegen, The Netherlands. Dr. M. Mulholland is at Philips Scientific, Cambridge, K. U. Dr. B. G. M. Vandeginste is at Unilever Research, Vlaardingen, The Netherlands. Computer Corner Contributions Contributions of between 400 and 900 words are wel- come in the following categories: hardware, software, chemical applications, mathematical tools and interfac- ing. Please send your papers either to: TrAC Computer Comer, D.L. Massart, Vrije Universiteit Brussel, Fakulteit der Geneeskunde en der Farmacie, Farmaceutische Scheikunde, Laarbeeklaan 103, B-1090 Brussels, Belgium. or TrAC Computer Comer, A.P. Wade, Department of Chemistry, University of British Columbia, 2036 Main Mall, Vancouver, B.C. Canada V6T lY6.