On Replacing Peer Review with Legal Challenge in Scientific Research: An Opinion PMlip J. Held Friedrich K Port, Robert A. Wolfe, Nathan W. Levin, and Marc N Turenne U W e r s H y of Michlgon Schools of Medicine and Public Health Ann Arbor, Michigan (PJH, FV, RAW, MNT) and Beth kad Medlcal Center, New York New York (NW) We appreciate the invitation from the editor of Seminars in Dialysis to express our opinions about a topic that is very acute in our personal and pro- fessional lives. In March 1993, Minntech Corpora- tion, which sells Renalina, filed a lawsuit in the United States District Court for the District of Maryland against the five authors of this editorial (plus one other person). Minntech is seeking over one million dollars in damages from us as individu- a l s based on our reports of scientific research find- ings. The suit claims damages to Minntech because our research had suggested an association of mortality nsks with the use of certain reuse agents, including Re&*, in conventional dialyzers in freestanding di- alysis units. Minntech subsequently publicized many of its allegations in letters to various journals ( 1 4 ) . Until now, in our pursuit of scientific objectivity, we have abstained from publishing any written re- sponse to what we view as an unfounded legal suit. Our primary goal to date has been to publish the scientific results through the peer review process and to let the renal community judge the merit of this research. In fact, at the time of this writing the scientific research paper in question has been ac- cepted and is in press at the American Journal of Kidney Diseases ( 5 ) . Minntech’s decision to pursue litigation has been a subject of comment in this and other journals. A recent editorial by C. M. Kjellstrand in a previous issue of this journal (6) describes some of Minn- tech’s allegations. Other publications have ap- plauded the right for a legal challenge to scientific research (7) or have provided possible mechanisms and discussion explaining the associations that we reported (8-10). These publications follow other re- lated research (1 1-15). Although we are in agreement with much of Dr. Kjellstrand’s editorial, there are two assumptions in Dr. Kjellstrand’s editorial that require clarification and correction. Furthermore, we believe that the largely unilateral publicity and the assertions by Minntech should not go unanswered. Dr. KJellstrand Suggests that the Research May Have Been Presented T o o H a s t i l y Quite the contrary is the case. At the time of our discussions with representatives of several Federal AddresJ conespondence to: Philip J. Held PhD. Univenity of hAichicpn. 315 West Huron--sUlte 240, Ann Arbor. MI 48103. Seminan in Dh&i.GVd 7. No 2 p l a r a ) 1994 pp 98-102 agencies, the research process regarding reuse ger- micides and patient outcomes had been underway for more than a year. The results of a pilot study had been reviewed by the Scientific Advisory Com- mittee of the USRDS in June and September of 1592. The national study that was the focus of dis- cussions and review with federal health agencies (our present study) followed up this earlier re- search, and analytical work had been intensely pur- sued for at least five months. After discussing and reviewing our findings with federal health officials, we were directed by the Health Care Financing Administration (HCFA, the funding agency) to meet with representatives from HCFA, the Centers for Disease Control and Pre- vention (CDC) and the Food and Drug Administra- tion (FDA). To further discuss these results, we presented these findings to epidemiologists, statis- ticians, and other representatives of these agencies at several meetings in September 1992. Throughout this process, we were instructed by the sponsors of the research (HCFA) that the results were to be considered confidential. After these meetings, the FDA concluded that it was appropriate to schedule an invited session on October 8, 1992. At the FDA’s request, the authors presented their research find- ings at that meeting. The findings were also pre- sented to the dialysis community at the Annual Meeting of the American Society of Nephrology (ASN) in November 1592. In short, the study was thorough and precise, with careful and deliberate attention to detail. The authors communicated with Minntech repre- sentatives on numerous occasions in October and November 1992 and also met with several Minntech executives. During this period, the authors ex- plained their analytical methods and results to Minntech, provided statistical output to Minntech, and considered Minntech’s disagreements with the research. As part of a prepublication review pro- cess, a draft version of the study manuscript was sent in February 1993 for comment to federal of% cials who had been involved in the previous review process. A copy was also sent to Minntech and, as is normal in the scientific process, copies were also sent to a few knowledgeable researchers in the renal community for review. Minntech did not provide comments on the draft and instead proceeded with a lawsuit. In the course of studying and reporting data with potentially significant public health implications, we have proceeded in a careful, responsible, and 98 ON REPLACING PEER 99 ethical manner. Suppressing, delaying, or otherwise withholding these findings from federal agencies or the dialysis community would have been wholly in- consistent with our obligations as researchers and with the interests of patients and dialysis providers. In our opinion, stopping or delaying publication of this paper and chilling open discussion of these find- ings have been major objectives of Minntech’s law- suit against the six authors. Dr. Kjellstrands Editorial Addressed Minntech’s Asserted Inability to Gain Access to the Underlying Data Used in our Analyses We agree with the principle that data upon which public health findings are based should be available for evaluation and assessment by the medical com- munity. At the same time, because much of these data involves medical information about identified patients, there is an obvious need for appropriate safeguards to protect the privacy of these patients. We have at all times willingly cooperated with both Minntech and federal health agencies in reconciling these interests. All researchers who work with patient data of the kind used in this research must, according to federal law (the Privacy Act, 5 U.S.C. Section 552 a [il 131, sign a confidentiality agreement not to release the data. The authors were therefore bound by federal law to keep the data confidential and not release them (Minntech was informed of this before it filed its lawsuit). Since the authors support the right of all researchers to carry out their own analysis of these data files, the authors met with representa- tives from Minntech and the Federal government in November 1992 to explain exactly how to go about gaining access to the data through submission of a research protocol. At Minntech’s request, the authors referred Minntech to a research firm that had had prior ac- cess to the federal data and could assist Minntech in obtaining and evaluating the data using the research protocol route that is open to all researchers. Minntech engaged this firm and notified its clients, the government, and the renal community that this firm, using the federal data, would be conducting its own analysis of the questions raised by our paper. Shortly thereafter, however, Minntech decided not to have this firm proceed with its plan to submit a research protocol, obtain the data, and conduct sta- tistical research. We do not h o w the basis for Minntech’s decision. We have seen no evidence that Minntech notified its clients or the renal com- munity of its change in plans. Minntech instead chose to pursue litigation against the United States (in addition to its separate litigation against the authors), seeking the underly- ing medical data under the Freedom of Information Act. For present purposes, we need not speculate on the merits of that case or on Minntech’s motives for pursuing litigation rather than following the more conventional research process for obtaining access to these data. It is clear, however, that we have in no way concealed the data used in our anal- yses from Minntech, and any suggestion to that ef- fect is untrue. To the contrary, we made substantial efforts to help Minntech understand our research, gain access to the underlying data, and engage in independent analysis. Minntech Has Incorrectly implied That There Was a Conflict of Interest and the Research Was Biased Against Minntech Minntech has also attempted to discredit the re- search findings by attacking the messengers. Minntech’s implication that any of the six research- ers was biased or had a conflict of interest is not supported by the facts. (Notably, while Minntech was asserting this conflict of interest, Minntech it- self repeatedly relied on research findings and pre- sentations by Alan Collins, MD, often without any mention of the fact that Dr. Collins had been a paid consultant for Minntech.) The study initially presented at the FDA and ASN meetings was for patients treated in freestand- ing units which used predominantly conventional dialyzers. This subpopulation of the total U.S. di- alysis population was chosen before any analyses were done for two reasons: (i) since the database had limited information on comorbidities (age, race, gender, cause of renal failure), patients in freestand- ing units were chosen as a more stable and more homogeneous population than if patients treated in hospital units had been included; and (ii) since high- flux dialyzers are rarely used without reprocessing, we considered it difficult to find an adequate control group of units employing only single use. Thus our design was rational, consistent with previous re- search designs, and not driven by bias or selection of the most striking results. Subsequent analyses of other patient groups, including patients in hospital- based dialysis units and patients in units using high- flux dialyzers, are also reported in the final publi- cation ( 5 ) . Regarding the authors and alleged conflicts of in- terest stemming from financial and other economic forces related to the issue of dialyzer reuse and Minntech, let us clearly state the facts. First, from the outset of this research, the authors never had a predisposition against dialy zer reuse. To the contrary, several of the authors had previ- ously published findings supportive of reuse ( 1 6 17). (Notably, Mmtech has cited some of these pro-reuse findings in its product literature; appar- ently these favorable findings were not considered the product of bias). Indeed, after our research was presented to the government and the ASN, and prior to the filing of the lawsuit, Dr. Levin wrote an editorial in this journal that was supportive of reuse and urged that our findings be interpreted cau- tiously (18). Dr. Levin’s editorial cautioned (as we have in all presentations of these research findings) that the associations found in our analysis should not be interpreted as showing a causal relationship between Rendin@ reuse and mortality. 100 Held et at Second, our dialysis reuse research has been funded exclusively by agencies of the United States Government. No corporation or other private orga- nization had any involvement in the research, with the exception of our request for comments from Minntech and another company that currently sells a glutaraldehyde-based germicide.’ Finally, as to other activities of the authors, Drs. Held, Port, and Levin have spoken at professional meetings sponsored by manufacturers of dialysis products for which they received a speaker’s hon- orarium and travel expenses. In the case of Drs. Held and Port, the sponsor was the Baxter Health Care Corporation; for Dr. Levin the sponsor was Fresenius USA, Inc. Baxter has provided research support to the University of Michigan, Dr. Port’s professional base; Fresenius AG, a German corpo- ration, has provided research support to Beth Israel Medical Center, the professional base of Dr. Levin. Neither this research support nor the speaking en- gagements had anything to do with dialyzer reuse activities. In addition, Dr. Wolfe has consulted with National Medical Care, Inc., on issues undated to dialyzer reuse. Beth Israel Medical Center has been involved in the development of dialyzer heat steril- ization, a method for the reprocessing of dialyzers. This research has been conducted at no financial advantage to Dr. Levin. N o research support has been paid to Dr. Levin or to Beth Israel Medical Center for dialyzer reprocessing research. Dr. Levin has not received consultant fees or any direct or indirect payment from Fresenius AG or Frese- nius USA, Inc. Dr, Levin owns less than YIO of 1% of the outstanding common stock of Fresenius USA, Inc., the shares of which are publicly traded. None of the authors have ever owned Minntech stock or had options to buy or sell Minntech stock. In reporting these facts, we go far beyond the disclosure that is required in even the most demand- ing research journals. (As noted above, we also go far beyond Minntech’s treatment of its direct finan- cial relationship with various physicians.) We do so because it is essential that our scientific findings be evaluated on their merits-not on the basis of un- founded rumor or personal attacks on the authors. The reality is that we were motivated to carry out and report this research solely by our interest in scientific research and in arriving at the correct sci- ence regarding the relative effectiveness of dflerent end-stage renal disease therapies. Even a strong b i a ~ could not have m&ed the epidemiologic and statistical analysis of over 60,OOO patient years of experience or the results of the study. The conclu- sions we reached from these data reflect our best professional judgment, and our analysis and conclu- sions have been extensively reviewed by profes- sional peers in the renal community as well as pro- fessional and governmental organizations. In presenting these findings in oral presentations and a forthcoming paper, we have been candid and forthright about the limitations of epidemiological studies, including this one. We repeatedly made considerable effort to emphasize that no causal link between use of a germicide and mortality risk could be inferred from these studies. We emphasized from the beginning that the variation in outcomes from unit to unit was large and that there were some dialysis units using all gemkides, including Rena- line, that had low mortaIity compared with the av- erage of units not reusing. In short, however much Minntech may dislike or disagree with our conclusions, we find its public effort to discredit these findings by attacking the authors to be without justification. It is irresponsi- ble and totally unfounded for Minntech to assert in a lawsuit and in letters to journals--as it has-that these researchers knowingly produced and dissem- inated false research. Germicide Efficacy Versus Effectiveness One of the points we have made repeatedly is that our associations cannot distinguish between the ef- fects of the germicide used in reprocessing and the manner in which the germicide is used. The latter might be called the “user factor.” Our research fo- cused on the “effectiveness” of reuse, which in- cludes both the “efficacy” of the product and the manner in which the product is used (user factor) in nonselected U.S. centers. Minntech has frequently made the claim that, if used as directed, RenalinQ is safe. Implicit in this claim is that Minntech is talking about “efficacy,” i n contrast to our focus on “effectiveness,” which includes “efficacy” as well as the “user factor.” Indeed, we have fnquently pointed out that our results may not have been caused by the germicide itself, but instead may be explained by the manner in which it is used. Even if a n t e c h is correct about the efficacy of R&u&n*, the statistical asso- ciations we have reported would still point to sub- stantial public health problems that may be present because of misuse of the product. Clearly the FDA policy response suspected a user problem since it focused on the methods of proper use of R e d i n @ . One of the major reasons for presenting and pub- lishing the results of studies such as ours is to allow the scientific community to examine the evidence and consider what mechanisms might be responsi- ble for the observed association. To bring up an analogy, if use of an efficacious drug were associ- ated with an increase in mortality in a substantial subgroup of patients it would certainly call for in- tense scrutiny. Failure to report such a finding would be considered unethical. It was in this spirit that these results were discussed and reviewed by the authors with the HCFA, the CDC, the FDA, and the renal community. Should such reports be suppressed for fear of lawsuits? In this regard, Dr. ON REPLACING PEER 101 Kjellstrand’s editorial makes a very good point that the current litigation will only slow down the pro- cess of finding the truth in this matter. What Are the Lessons of this Experience for Research? We did not invite or welcome the litigation from Minntech. At the direction of federal sponsors and a scientific advisory committee, we diligently en- gaged in research addressing a significant public health issue and reported the findings accurately and responsibly to federal health agencies and the dialysis community. Minntech’s decision to resort to litigation in which it seeks more than one million dollars in al- leged damages from the investigators has diverted too much energy and money from research and has inserted fear of litigation into the scientific process. The defendants’ legal costs have been very substan- tial over the last year, even before entering the de- position phase. Trial is not likely until late 1994. One can only speculate what the ultimate expense of this litigation will be. We have been fortunate to have support from our institutions, which have not flinched at this challenge to the research process. What would have happened if we had not been so fortunate? What would happen to other researchers not as well supported by their institutions? And more importantly, what does the precedent of this lawsuit mean for the research process? At least one other research group has federal support to examine dialyzer reuse. Try to imagine the climate they face as they prepare to report their findings. Dr. Kjell- strand’s editorial provided a good summary of what this lawsuit means for the research process. There are at least two qualities of the legal system that are very detrimental to the research process. First, resolving legal disputes is costly and ex- tremely time-consuming; second, the legal system is not an efficient machine to find truth in complex scientific matters. The fact that our institutions have agreed to pay for legal fees so far does not make this litigation “free” to society. Just as the cost of medical mal- practice is passed on in higher prices to consumers and taxpayers, the cost of litigation in research will be passed on to others. This country does not need to have a substantial new cost burden in our current health care system. Just when our medical system is focusing on more studies of outcomes research such as our own project for the goal of a more efficient health care system, a new wrench is thrown into the works. Scientific issues will henceforth be threat- ened by the courts! What if errors are made in scientific research? As Dr. Kjellstrand noted, this will sometimes occur, but scientific processes exist to correct such errors. We suspect that this process is robust enough to deal with errors of many types, even those that could arise from conflicts of interest that are present in industry-sponsored research. One process is the manner in which studies are repeated and reproduced as part of scientific in- quiry. We believe that Minntech has the right to the federal data under the same conditions with which we and all researchers have had t o comply. Minntech should be able to confirm or challenge our results. There are established processes for re- searchers who want to gain access to the data, pro- cesses that assure appropriate confidentiality. In- stead of relying on a research proposal, however, Minntech has resorted to much more heavy-handed attacks, which pose different threats to society. Sci- entists differ with each other often. To challenge published research, a scientist typically builds an- other experiment with other data, other methods, a n d o r other theoretical models. Over time, the er- rors of the past are bypassed, refuted, and replaced by more complete and accurate results. Our advice to Minntech before the litigation was to take the high road. This approach might have included making a public statement saying i t thought these six researchers were wrong and that Minntech was going to show them to be wrong. We further advised them that if their investigation found that there is something to these associations and that the problem lies with Minntech products or how they are used, Minntech as a reputable and respected company should have fixed matters. That advice still stands, but we now believe that Minntech should also repair some of the damage it has done to the scientific research process by reim- bursing the Urban Institute, the University of Mich- igan, and Beth Israel Hospital for legal expenses. The current message Minntech has given to the re- search community is that those who report adverse facts about experience with Minntech products may suffer legal expenses from litigation and unfounded accusations in the literature. Failure to reverse these actions and reimburse the expenses already incurred would stand as a signal to those who might naively believe that they could comment on their experience with Renalinm in a free exchange of facts. Our scientific advice to Minntech at the initial presentations was to test whether the reuse proce- dures with Renalina differed between units with high and low standardized mortality rates. We pro- posed that we identify dialysis units using RenalinB with the highest and lowest standardized mortality rates and blind Minntech to their grouping. On-site data collection and analysis had a very high proba- bility of determining whether the “user factors” may be an explanation for the observed differences in mortality. It was our belief that this study could have been completed in early 1993. To our knowl- edge, this advice was not followed and the study was never implemented. The data we used were for 198!3-1990. We hope that if the “user factor” was the cause of the ele- vated mortality that our study might have already led to improvements in practices. It appears that the FDA notice had that intent. 102 Held et at To those readers who have heard so much rumor and innuendo about the authors that they suspect that the authors must have done something to de- serve such litigation we offer the following analogy: “Henry, why are you here?” Ralph Waldo Emer- son is supposed to have asked Thoreau, who was in jail for refusing to pay war taxes. “Waldo, why are you not here?” Thoreau replied. The current litigation from Minntech must not and will not deter us and others from continuing to researcb dialysis-related issues and report scientific findings with potential public health implications. We do invite and welcome scientific questions and challenges. We clearly see a need for continued, quality research in this area to identify the reasons for the observed differences in mortality. Research findings are rarely an endpoint. Our studies provide new findings and answer some questions while rais- ing new questions which need to be addressed in future research for the pursuit of truth. We hope that the medical community wili demonstrate their defense of the scientific process through condem- nation of inappropriate litigation. We hope that Minntech will take tangible action to repair the damage that their litigation has done t o the scientific process and t o show their commitment to preserv- ing scientific inquiry. Note Added in Proof On February 15, 1994, very close to the time of writing of this editorial, Minntech filed a research protocol to obtain the data from the federal govern- ment. I. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. IS. 16. References ANON: Minntefh files new lawsuit. Dial 7ransplant 22232. 1993 Neumann M E Minntcch sues authors of new report on dinlyzer re- use. Neph News Issues 7(5):8, 10; 1993 ANON: Scientists who blow the whistle on accepted medical treat- ments race a new eaemy-They m y become defendants in a lawsutt. Partners in Prevertioa Spring5, 1993 ANON: Minntech fiks lawsuit @nst “Renalin* Paper” authors. Contemp Dial Nephrof I4(6):13, 35; 1993 Held PJ. Wolfe RA, Gaylin DS. Port FK, Levin NW, Turcam MN: And- of the r s J o c i D t i of dialper MX practim and patient Wtcomcs. Am I Kidney Dis (In press) yieUstraod CM: The search for scientifi truth versus eorporatc in- tcruts. Semin Dial 6:2111-283,1993 Depalma J: Law and truth, or all chokes involve compromise. Neph News Issues 7:3&39, 1993 Desai S: Tbc pitfalls of sterilants in the dialysis unit. Neph News Issues 7(11):12. 31, 34-35; 1993 Ward R4: Reaction. Neph News Issues 734-3s. 1993 Millot CW: Re~etiw. Neph N e w s Issues 734. 36; 1993 Tokam JI, Alter MJ, Faveio MS. Moyer LA. Bland LA: National surveilknee of dialysis-associated disensn in t Q United States. ASAIO J 3%%6-97S. 1991 Flahcrty IP, Garcia-Houchins S. Chudy R. Amow PM: An outbreak of OlPmacpPcivt bacteremia traced to contamiaated O-rings in =pro- cessed d i a l y w . Ann Intern Med 1 1 9 1072-IQ78,1993 b l a n G , Reingold AL. Carson LA, Sikox VA. Woodky CL. Hayes PS, Hightower AW. McFarland L. Brown JW 111. Pctencn NJ. Favero MS, Good RC. B ~ o w n CV: infections with Myedwcterium rheloari in patients receiving dialysis and using r e p ~ ~ ~ c l s a f bemodi- alyzen. I Infect Dk 1SZ1013-1019, 1985 LMy PW. &ck-Saguc CM. BIand LA, Agmw SM. Arduino MI. Mmuth AN. Munay RA. Swcnson JM, Jarvis WR: MrcobPrtcrium rhelond infection among patients receiving high-nwr dialysis in a hernodialysis clinic in California. J Iderr DU 161:8S-W. 1990 United States General Accounting Office: Hospitd Sferiionts: Insuf- fk5ent FDA Regulation May Pose a Public Health Risk. GAO-HRD- 93-79. W a s k i t o n DC. 1993 Held PJ. h u t y MV, DiDmoDd LH: SuMval analysis of patients un- d e m i n n dialvsis. JAMA 257:MW. 1987 17. & x i a h f& the Aduanccmm of MniicPl instntnrentptioa: A A M I Recomnendrd Practice. Reuse of Hemodiaiyrers (AAMI ROH- 1986). Arlington, VA, 1993 (Nathan Lcvm was co-choinnan of the 18. Levin NW: D d y z c r reuse: A currently acceptable practice. Semin Dial689-90, 1993 canmittcc that produced *is report)