key: cord-0425791-qbcwjqcy
authors: Baraldi, J. H.; Picozzo, S.; Arnold, J.; Volarich, K.; Gionfriddo, M.; Piper, B. J.
title: Accuracy of Conflict-of-Interest Disclosures of Physician-Authors Publishing in High-Impact U.S. Medical Journals: A Comparison of the Journal of the American Medical Association and the New England Journal of Medicine
date: 2021-09-15
journal: nan
DOI: 10.1101/2021.09.12.21263468
sha: 2d135f0586da6bce26e378483a0c61d3ab857363
doc_id: 425791
cord_uid: qbcwjqcy

Objective: To assess the accuracy of self-reported financial conflict-of-interest (COI) disclosures in the New England Journal of Medicine (NEJM) and Journal of the American Medical Association (JAMA) within the requisite disclosure period prior to article submission. Design: Cross-sectional investigation. Data Sources: Original clinical-trial research articles published in NEJM (n = 206) or JAMA (n = 188) from January 1 to December 31, 2017; self-reported COI disclosure forms submitted to NEJM or JAMA with the authors published articles; Open Payments website (from database inception; latest search: August 2019). Main outcome measures: Financial data reported to Open Payments from 2014 to 2016 (time period that included all subjects requisite disclosure windows) were compared to self-reported disclosure forms submitted to the journals. Payments were defined as those not associated with a research study or formal research funding. Payment types were categorized as disclosed, undisclosed, indeterminate, or unrelated. Results: Thirty-one articles from NEJM and 31 articles from JAMA met inclusion criteria. The physician-authors (n = 118) received a combined total of $7.48 million. Of the 106 authors (89.8%) who received payments, 86 (81.1%) received undisclosed payments. The top 23 most highly compensated received $6.32 million, of which $3.00 million (47.6%) was undisclosed. Disclosure rates were the equivalent between the top 23 and the entire sample. Conclusions: High payment amounts, as well as high proportions of undisclosed financial compensation, regardless of amount received, comprised potential COIs for two influential US medical journals. Further research is needed to explain why such high proportions of general payments were undisclosed and whether journals that rely on self-reported COI disclosure need to reconsider their policies.

Financial conflicts of interest (COIs) are a perennial problem for medical research and practice. 1, 2 Physician researchers who receive industry payments are more likely to demonstrate results favorable to the companies funding them; 3, 4 are more likely to prescribe drugs and use of medical devices produced by these companies, from statins 5 to opioids 6 to endoscopic 7 and orthopedic devices; 8 and they may unduly influence other physicians by contributing to research that others use to guide their own clinical practice. [9] [10] [11] [12] [13] [14] [15] [16] Industry payments to physicians therefore may bias healthcare providers' delivery of evidence-based medicine and interfere with their responsibilities to their patients. (ICMJE) has produced its own COI form to help medical journals maintain COI disclosure standards for physician-authors seeking to publish articles in peer-reviewed medical journals. 18 Many journals have adopted the use of this form, requiring authors submitting manuscripts to them to disclose payments received from manufacturers of products related to the article content in the 36 months prior to submission. 19 These author disclosures can be verified by viewing the 3 2017 the Journal of the American Medical Association (JAMA) published an issue dedicated to the subject of COI disclosures to highlight the multifaceted nature of the problem. 29 NEJM and JAMA are the peer-reviewed general medical journals published in the United States with the highest and second-highest impact factors, respectively. Both journals publish with similar frequency (weekly for the former and 48 times per year for the latter), emphasize publication of original research as well as reviews, are popular for physician-authors, and publish articles that receive wide coverage both within the scientific community and in the popular news media. The impact and reach of these two journals have substantial potential to shape future research and patient care. To date, there has been no comprehensive study of COI disclosures in these two journals. This study examined COI disclosures among physician-authors who published articles in either (or both) of these journals in 2017, the first year for which complete data exist for the earliest possible disclosure period following the inception of Open Payments.

Identifying patterns of disclosure transparency at the beginning of the existence of Open Payments revealed the extent to which physician-authors publishing in NEJM and JAMA follow COI disclosure policies.

Original research articles (n = 394) detailing the results of randomized clinical trials and published in NEJM (n = 206, 52.3%) and JAMA (n = 188, 47.7%) from January 1, 2017, to December 31, 2017, were examined. The first and last author of each article were identified.

Articles were excluded from further examination if either the first or last author did not have an M.D. or D.O. degree and if either author did not have a record in Open Payments. These authors had to be physicians with Open Payments records. This is due to this study's objective of exploring COI disclosures of physician-authors by comparing Open Payments records, which consisted of disclosures made by the companies making the payments, with self-disclosures made by the authors to the journals. These self-disclosures consisted of the authors' . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted September 15, 2021. ; https://doi.org/10.1101/2021.09.12.21263468 doi: medRxiv preprint identifications of the companies that paid them, not of the amounts, which the companies themselves provided to Open Payments.

Data reported to Open Payments from 2014 to 2016 were compiled and compared to selfreported disclosure forms that had been submitted to NEJM or JAMA with the authors' published articles. Data collected about the authors included sex, specialty, journal(s) of publication, and yearly payment information. Open Payments defines "payments" as either general "payments that are not associated with a research study" or research payments that "are associated with a research study." 30 This study focused solely on general payments, which included compensation for promotional speaking, consulting, travel and lodging, food and beverage, honoraria, and current or prospective ownership or investment interest.

The articles were examined to determine their areas of investigation (e.g., cardiovascular disease, diabetes, cancer). This occurred with reference to the title, key words, abstracts, and content of each article. Three co-authors of this study collected the data, resolving by discussion any disagreement in interpretation of article topics for the purpose of disclosure analysis. Each article's area of investigation was compared against the product portfolios and research pipelines of the companies that paid the physician-authors, according to their Open Payments data.

Payments from companies disclosed by the author were labeled as "disclosed" for this study.

Payments from companies not listed on the respective authors' disclosure forms were investigated further and categorized as "undisclosed," "indeterminate," or "unrelated."

Occasionally, a company that did not match the disclosures on the author's form was later determined to have made a disclosed payment. For example, a company with multiple names was determined to have made a disclosed payment under only one name, prompting a review that revealed the fact that the company's payment had been disclosed, just under a name that was not immediately recognizable on the disclosure form. Every payment from a given company was analyzed by researching that company, thus putting all payments from that company under one umbrella for the purpose of COI disclosure. For example, if a company produced a drug related to the content of an author's research article, then every general payment that that company made . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted September 15, 2021. ; https://doi.org/10.1101/2021.09.12.21263468 doi: medRxiv preprint to the author, regardless of the nature of the payment, was construed as a COI. Payments were considered for both the parent and subsidiary companies.

The criteria in Table 1 were followed to categorize payments as "disclosed," "undisclosed," "indeterminate," or "unrelated." These definitions were adapted from the ICMJE disclosure form used by both journals, which states:

[Authors] should disclose interactions with ANY entity that could be considered broadly relevant to the work. For example, if your article is about testing an epidermal growth factor receptor (EGFR) antagonist in lung cancer, you should report all associations with entities pursuing diagnostic or therapeutic strategies in cancer in general, not just in the area of EGFR or lung cancer. 31 ICMJE requires disclosures for 36 months prior to submission. Therefore, this study focused on all payments within 36 months of the submission dates. A copy of the ICMJE form reviewed for this study is available in the Supplementary Materials. 31 Payment data in the study window, including company name, amount, and purpose, were manually extracted from the Open Payments database into a spreadsheet accessible to all authors of this study. Payments were then categorized based on the ICMJE guidelines (Table 1 ) by three co-authors in their first two years of medical school and disagreements were resolved by discussion among these raters. NEJM provided the disclosure forms as attachments to their articles; JAMA provided a list of disclosures at the end of each article. The NEJM articles investigated in this study stated their submission dates or included this information on the author disclosure forms, which are made available to the public as attachments to each article. For JAMA the submission date was approximated by using the date when the article was published.

JAMA's official position is that, as of 2016, the median time from article submission to acceptance was eighteen days, and the median time from acceptance to first online publication another fourteen days, roughly totaling one month. 32 Therefore, unknown submission dates were estimated as 30 days prior to respective publication dates; this caveat is important for data . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted September 15, 2021. ; https://doi.org/10.1101/2021.09.12.21263468 doi: medRxiv preprint interpretation. It was assumed that a COI encountered within these thirty days would be unlikely to influence the manuscript, presumably already written in its nearly final form. Descriptive statistics were calculated. ROUT analysis (maximum desired false discovery rate Q = 1%) identified outliers. The Wilcoxon rank-sum test assessed the extent of parity between distributions. The flowchart was generated using Lucidchart (Lucid Software Inc., South Jordan, UT, USA). A p-value of p < 0.05 was considered statistically significant. Mean amounts are presented ± standard deviation.

A total of 394 original research articles published in NEJM (n = 206) and JAMA (n = 188) from January 1, 2017, to December 31, 2017, were examined. Thirty-one articles from NEJM and 31 articles from JAMA met all criteria for inclusion, with a total of 118 unique authors.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint Figure 1 ). Authors in aggregate received comparable total payments as well as similar total payments by category as proportions of total payments by journal ( Table 2 and Figure 2 

Of the 106 authors who received payments, 55 made disclosures of which the disclosed companies reported dollar amounts that summed to at least half of the authors' total payment . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted September 15, 2021. ; https://doi.org/10.1101/2021.09.12.21263468 doi: medRxiv preprint amounts. Twenty had a three-year disclosure rate of 100%; ten of these published in NEJM, and the other ten published in JAMA. The other 35 authors who disclosed at least half of their payments had disclosure rates that ranged from 54.5% to 99.9%. Of the 51 authors who disclosed less than half of their payment amounts, eighteen disclosed between 0.007% and 42.3%. 33 authors who received payments disclosed 0%, or no amount, of their payments received. Of the authors who disclosed 0%, 21 of them published in JAMA, and twelve published in NEJM ( Figure 3 ).

Subgroup analysis further examined the disclosure rates of all physician-authors who received compensation by comparing the outliers with the non-outliers. A Wilcoxon rank-sum test found no significant difference (p = 0.1849) between these two subgroups. The median disclosure rate of outliers was 78.1% and of non-outliers was 33.4%.

Sensitivity analysis compared the disclosure rates of all physician-authors who received compensation with those compensated who were not outliers. A Wilcoxon rank-sum test found no significant difference (p = 0.6406) between these two groups. The median disclosure rate of all those who received compensation was 68.1%, while the median disclosure rate of nonoutliers was 33.4%.

A Wilcoxon rank-sum test also found no significant difference in COI disclosure rates between NEJM and JAMA authors (p = 0.0849). The median disclosure rate of NEJM authors was 79.4% and of JAMA authors was 3.50%.

Little variability was observed across the individual years that fell within the 36-month 

The physician-authors in this study represented 33 distinct medical specialties. Fourteen (42.4%) of these specialties were subspecialties of internal medicine. The two most-represented specialties among the 118 physician-authors were cardiovascular disease (n = 16) and general internal medicine (n = 11). Ten specialties each were represented by four to nine individuals, seven specialties each were represented by three individuals, five specialties each by two individuals, and eleven specialties each by a single individual (Supplementary Materials).

These novel data from highly influential US general medical journals (NEJM and JAMA) extend a sizable evidence base that has raised doubts about whether self-reported financial disclosure is a trustworthy mechanism for point-of-care databases, 13 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted September 15, 2021. ; https://doi.org/10.1101/2021.09.12.21263468 doi: medRxiv preprint COI disclosure rules and procedures venture to mitigate the impact of COI bias on the integrity of published manuscripts. This premise means that ascertaining the impact of payments on researchers, or how payments influence those receiving them, may help to delineate the process of this insult to publication integrity. Key to identifying such bias in the first place is examination of COI disclosure accuracy. This was the purpose of the present study. Accordingly, the authors of this study take no position on the physician-authors' intentions in non-disclosure of their COIs; we characterize the issue as a "process problem" rather than a "people problem," especially in light of the patterns observed in COI disclosure rates regardless of the journal's disclosure process and regardless of payment amount.

It was hypothesized that different disclosure processes between the two journals could produce different patterns in payment distributions and disclosure rates. This hypothesis derives from the fact that NEJM provided a copy of the original disclosure, while JAMA provided a list of disclosures. On the contrary, the data demonstrate no such significant differences between the two journals. Both journals had higher payment concentrations amongst the outliers than amongst the rest of the sample, illustrating a Pareto-principle pattern reflected across the two journals. This lack of differences in payment distributions and disclosure rates, despite a difference in the disclosure process, may imply that the journals' differing disclosure processes had no effect on payment distributions and disclosure rates.

Further analysis elucidated surprising parity between the outliers and the entire study sample.

Subgroup analysis indicated that physician-authors were just as likely to disclose COIs if their payment amounts did or did not constitute an outlier, and sensitivity analysis indicated that nonoutliers were just as likely to disclose COIs as the entire group of physician-authors who received compensation. Therefore, COI disclosure rates remained consistent regardless of total payment amount, answering the question posed by previous researchers of the difference that payment amount makes in COI disclosure. 55 This suggests that adherence to, or non-compliance with, COI disclosure procedures was not a function of payment amount, even for the outliers. It follows that targeting for correction the COI disclosure practices of those receiving the highest payment amounts would not address the fundamental problem of COI non-disclosure that this . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this this version posted September 15, 2021. ; https://doi.org/10.1101/2021.09.12.21263468 doi: medRxiv preprint study finds to be common to physician-authors who have received general-payment compensation.

A future study comparing COI disclosure patterns with this earliest period of Open Payments data might show a change in such patterns and, possibly, the effect of Open Payments on COI disclosure transparency. Future research may be more robust now that the SUPPORT Act has expanded the range of researchers whose data are collected by Open Payments. 56 As of January, 2021, physician assistants, clinical nurse specialists, certified nurse midwives, certified registered nurse anesthetists, and anesthesiologist assistants have entries on the Open Payments website. 57

This new data source may help to assess whether the observations of this study are applicable to mid-level healthcare providers.

The major limitation of this study is that of generalizability, especially of the findings to journals other than NEJM and JAMA. The authors assessed the data within a non-parametric analytical framework because there is no methodological justification for making inferences about payment distribution patterns amongst the broader community of physician-authors. The 118 physicianauthors that met inclusion criteria are not presumed to represent this broader community, despite the more general-interest nature of the content that NEJM and JAMA tend to publish. Moreover, the authors who publish in these two journals may more likely receive funding than those who publish in other journals generally. This suggests that the data may be skewed towards those who already receive more compensation, although this study's findings of no difference in disclosure rates between outliers and non-outliers may belie this point. Another limitation was small sample size across the two journals, which may have obscured possible differences due to low statistical power.

The fact that the preponderance (81.1%) of physician-authors in this study received payments that they did not disclose but that they nonetheless were supposed to disclose as COIs per ICMJE . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this this version posted September 15, 2021. ; https://doi.org/10.1101/2021.09.12.21263468 doi: medRxiv preprint guidelines and journal requirements demonstrates that these disclosure requirements in conjunction with the expectation of COI self-disclosure have been inadequate to ensure full COI transparency in either NEJM or JAMA and regardless of general payment amount received.

Making industry payments a matter of public record in the form of Open Payments presumes to mitigate this problem of COI disclosure opacity. Readers are encouraged to "trust, but verify" and to compare self-reported with industry-reported disclosures early in the investigative process.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. 

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint 1 . Distribution of total payment amounts compared between NEJM and JAMA. NEJM authors had a higher median payment amount, but JAMA authors had a higher mean. Distribution by COI disclosure rate (analysis not shown) followed a similar pattern. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted September 15, 2021. ; https://doi.org/10.1101/2021.09.12.21263468 doi: medRxiv preprint FIGURE 2. Payment amounts by category compared between NEJM and JAMA. Percentages shown are of the labeled category with respect to overall payments to those authors publishing in the respective journals. For more detailed information see Table 2 . . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted September 15, 2021. ; https://doi.org/10.1101/2021.09.12.21263468 doi: medRxiv preprint FIGURE 3. Flowchart of payment disclosure rate distributions. Of the 35 authors who disclosed at least half (but not 100%) of their payment amounts, the range of actual disclosure rates was 54.5 to 99.9%. Of the eighteen authors who disclosed less than half (but not 0%) of their payment amounts, the range of actual disclosure rates was 0.007 to 42.3%.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint

The copyright holder for this this version posted September 15, 2021. ; https://doi.org/10.1101/2021.09.12.21263468 doi: medRxiv preprint

Ten years later: a review of the US 2009 Institute of Medicine report on conflicts of interest and solutions for further reform

Scope and impact of financial conflicts of interest in biomedical research: a systematic review

Conflicts of interest among committee members in the National Academies' genetically engineered crop study

Industry sponsorship and research outcome

Association of industry payments to physicians with the prescribing of brand-name statins in Massachusetts

Association of pharmaceutical industry marketing of opioid products with mortality from opioid-related overdoses

Assessment of pharmaceutical company and device manufacturer payments to gastroenterologists and their participation in clinical practice guideline panels

US makers of joint replacements are fined for paying surgeons to use their devices

Conflicts of interest and disclosure in the American Psychiatric Association's Clinical Practice Guidelines

Quantification of conflicts of interest in an online point-of care clinical support website

Financial conflicts of interest in clinical practice guidelines: a systematic review

Gestion des conflits d'intérêts durant l'élaboration de lignes directrices en santé

Analysis of conflict of interest policies among organizations producing clinical practice guidelines

ICMJE. Journals stating that they follow the ICMJE Recommendations

Undisclosed payments by pharmaceutical and medical device manufacturers to authors of endoscopy guidelines in the United States

Research misconduct: the poisoning of the wells

Science and the Media: Delgado's Brave Bulls and the Ethics of Scientific Disclosure

Institutional conflicts of interest and public trust

Delicate operation: how a famed hospital invests in a device it uses and promotes

Drug interactions: financial ties to industry cloud major depression study

Editors of The Lancet. Addendum: competing interests and the origins of SARS-CoV-2

Who are the COVID investigators?

Under-fire Lancet admits conflict of interest on lab-leak letter

The vexing problem of guidelines and conflict of interest: a potential solution

Conflicts of interest and the quality of recommendations in clinical guidelines

Conflict of interest and medical journals

Mississippi Valley Medical Association. A medical editor's resignation

Journal editor quits in conflict scandal. The Scientist

Publishing: The peer-review scam

Payments to physicians: does the amount of money make a difference?

Public Law 115-271 -Oct