key: cord-0712423-iexx4ohy
authors: Piaserico, Stefano; Gisondi, Paolo; Cazzaniga, Simone; Di Leo, Sara; Naldi, Luigi
title: Assessing the risk and outcome of COVID-19 in patients with psoriasis or psoriatic arthritis on biologic treatment: a critical appraisal of the quality of the published evidence
date: 2021-08-04
journal: J Invest Dermatol
DOI: 10.1016/j.jid.2021.04.036
sha: 0fa2b11d0d7903ff2d2c035e87dfa14bff7f64ee
doc_id: 712423
cord_uid: iexx4ohy

The need to rapidly spread information about the risk of COVID-19 in patients with psoriasis (Pso) and psoriatic arthritis (PsA) on biologics may have hampered the methodological rigor in published literature. We analysed the quality of papers dealing with the risk and outcomes of COVID-19 in patients with Pso and PsA receiving biologic therapies. The Newcastle-Ottawa Scale (NOS) was used to estimate the quality of the published studies. Moreover, to better contextualize results, specific internal and external validity items were further considered, i.e. case definition, modality of COVID-19 assessment, evidence for self-selection of participants, percentage of dropout/nonparticipants, and sample size calculation. Twenty-five out of 141 papers were selected. The median NOS score was 47% for Pso and 44% for PsA, indicating an overall high risk of bias. 37% of Pso and 44% of PsA studies included patients with suspected COVID-19 without a positive swab. No studies provided a formal sample size calculation. A significant risk of bias in all the published papers was found. Major issues to be considered in future studies are: reduction of ascertainment bias, better consideration of non-response or participation bias, and provision of formal statistical power calculation.

The severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic spreads globally since early 2020 with major health and economic consequences.

After a considerable amount of time, there is still some concern among dermatologists regarding a potentially increased risk of infection and/or worse outcome among COVID-19 patients on biologic therapy for psoriasis (Pso) or psoriatic arthritis (PsA).

Indeed, available data point to an increased risk of respiratory infections in patients being treated with antagonists of interleukin (IL)-17 and tumor necrosis factor (TNF)-alpha (Ford et al, 2013; Wan et al, 2020) . On the other hand, in COVID-19 patients uncontrolled inflammatory innate responses and impaired adaptive immune responses may lead to tissue damage, both locally and systemically. Many inflammatory cytokines appear to be involved in this phenomenon, including Tumor Necrosis Factor (TNF) and interlukin (IL)-17 (Feldman et al, 2020; Pacha et al, 2020) .

Overall, whether biologics enhance the risk or protect from development of severe or if Pso/PsA per se is associated with a more severe course of infection is yet to be ascertained.

Several papers have been published aiming to elucidate the risk of patients with Pso or PsA being treated by biologics during the COVID-19 pandemic.

However, the need to rapidly spread information to the dermatologic community may have hampered the methodological rigor in the currently published literature.

In this study, we analysed the quality and possible limitations of published studies on the risk and outcome of COVID-19 in patients with Pso or PsA receiving biologic therapies and make suggestions for future research studies.

The initial database search yielded a total of 141 items from PubMed and 53 from Embase (187 records after duplicate removal). 134 studies were further excluded based on abstract review because they did not match the inclusion criteria. After screening the full text of remaining 53 articles, 25 studies were eligible for the final qualitative assessment (Figure 1 ).

The characteristics of the selected studies and the respective quality scores according to the NOS scale are summarized in Table 1 . Further details are included in Supplementary Table   S1 for Pso and Table S2 for PsA.

The median NOS score was 47.2% (Interquartile range IQR=39.7-55.6) for Pso and 44.4% (IQR=40.0-55.6) for PsA, indicating an overall high risk of bias. No study reached a score ≥75%. In particular, no study satisfied the comparability items ("study controls for the most important factors" or "for any additional factor"), and, in cross-sectional studies, a sample size was never justified and response rates were provided in 0% of Pso studies and 22.2% of PsA studies.

Two out of 16 (12.5%) papers providing data on Pso combined the dermatological condition with other diseases. Seven (43.7%) studies on Pso did not include a control group. Six (37.5%) studies included patients with reported or suspected COVID-19 without a positive swab (table 2) . Thirteen out of 16 (81%) studies collected data on group of patients from a target population (typically the cohort of patients followed in one or more reference centres for Pso) and not according to self-selection by the patient (e.g., internet based surveys), but only 2 (12.5%) reported the rate of non-participants or dropouts (table 2) .

All of the papers on PsA also included patients with other rheumatological conditions, and only in one paper we could clearly distinguish the data regarding PsA patients from the other groups of patients. Two out of 9 (22.2%) studies on PsA did not include a control group.

Four out of 9 (44.4%) studies enrolled patients with reported or suspected COVID-19 without a positive swab. Two (22.2%) of the studies collected data based on self-selection by the patient, and one of these reported a 54% rate of non-participants or dropouts (table 2) .

No studies on Pso or PsA provided a statement concerning the power of the statistical tests used or a formal sample size calculation for incidence or prevalence estimates.

A detailed description of the studies according to the additional elements described in the Method section is summarized in table S3 and table S4 for Pso and PsA, respectively.

In this comprehensive meta-research comprising 25 studies on the risk and outcomes of COVID-19 in patients with Pso and PsA, we found a high risk of bias in all of the published papers. No study reached a NOS score ≥75%.

One common flaw was the lack of a suitable comparator group. Seven out of 16 (43.7%) studies on Pso did not consider any control group. This prevented a proper evaluation of the risk of COVID-19 associated with the disease or its treatment.

One major problem in 7 out of 25 (28%) studies, particularly registries, which collected data on COVID-19 patients and compared the proportion of patients who were hospitalised or died according to different treatments received for their underlying disease, was the lack of a reference to the underlying at-risk population, i.e., the population of patients treated by J o u r n a l P r e -p r o o f different medications from which the COVID-19 cases originated. This flaw is sometimes referred to as "floating numerators." It can be easily demonstrated, for example, that similar proportions can originate from underlying populations with largely divergent risks (Naldi and Cazzaniga, 2020) . Hence, the lack of information on the appropriate denominator (i.e., the source population) for COVID-19 cases, does not allow for calculating proper incidence rates and risks.

In 40% of the studies, patients with suspected COVID-19 were included without a positive test or a definite diagnosis. When assessing the incidence rate of a disease with a low number of collected cases, this could determine a substantial impact on the analysis. 

Regrettably, only 16% of the studies reported the percentage of responders on the overall group of potential participants. This may represent a relevant issue (Non-response or participation bias), especially in telephone-and web-based studies.

Web-based non-response might be related to technological difficulties. Since Internet access tends to be correlated with age, and COVID-19 severity is markedly greater in elderly people, web-based data could provide biased results.

On the other hand, also telephone-based collection of data may severely bias the analysis of data in an opposite way. If the surveys were conducted during business hours, active workers would be less likely at home compared to elderly retired individuals.

This potential mismatch between the characteristics of respondents in a non-random sample and those of the general population can lead to severe issues in assessing the outcome of interest. We acknowledge that removing non-response bias from a study may be an impossible effort. Regardless, researchers should declare the response rate of the overall population.

Furthermore, studies were mostly conducted in referral hospital centers and no populationbased studies were published (selection bias). Patients with Pso seen in referral centers are likely to have more healthcare exposure than the general population (ascertainment bias). In addition, unmeasured confounders from physician (e.g., collecting only a portion of COVID-19 patients) or patient (e.g., greater application of social distancing measures and personal protection strategies compared with the general population) may also have biased the analysis.

It should be recognized that several studies were well-constructed, with a proper calculation of the incidence rate of COVID-19 infection and COVID-19-related hospitalization and death, but none of them performed a sample size estimation and were most likely underpowered to detect any difference between psoriatic patients and the control group (type II error). For example, the study which included the largest available cohort of Pso patients for Covid-19 outcomes, was able to reach a maximum power of only 64% (exact binomial test). This was largely due to relatively low incidence rates in both the population and the study cohort (Gisondi et al, 2020c) .

Our observations are in line with similar data from quality surveys in other clinical areas. A study showed that the quality of papers published in the New England Journal of Medicine, J o u r n a l P r e -p r o o f Lancet, and JAMA was lower in the first months of 2020 as compared to the same period in 2019 and that the decline could be attributed to COVID-19 (Stefanini G, 2020) . Fewer studies were randomized in 2020 than in 2019 (29.2% vs 41.4%; OR 0.58; 95% CI 0.41-0.82).

Additionally, according to GRADE criteria used in the paper, just 13.7% of 2020 studies were considered "high quality," compared with 27.6% of studies published in 2019 (OR 0.41; 95% 0.27-0.63). In a sensitivity analysis that excluded COVID-19 research, no difference was found between the quality of original research published in 2020 compared with the year before.

Another study confirmed that COVID-19-related research in the same journals (i. e. the New England Journal of Medicine, Lancet and JAMA) was of lower quality than research on other topics in the same journals for the same period of time, with a great effect size (Zdravkovic et al, 2020) . Interestingly, the number of publications on COVID-19 alone was almost the same as the number of publications on all other topics.

There are many reasons that could explain the high frequency of biased published studied.

COVID-19 is (still) an unknown disease and there was an urgent need to collect and publish some data. Everybody agreed on the fact that little, even flawed, data were better than no data.

Against this backdrop, traditional peer-review system has been stressed by the enormous number of COVID-19 related manuscripts (Bauchner et al, 2020) .

Several studies were similar, and redundancy in COVID-19 studies may have led to lost time and energy for research teams, scientific journals and reviewers (London and Kimmelman, 2020 ).

Our study is not without limitations. We evaluated the methodological quality (i.e., internal quality) of existing studies using NOS, a well-established and widely used score system, but J o u r n a l P r e -p r o o f not completely appropriate in case of studies lacking a formal design. Indeed, NOS has been criticized by some authors (Stang, 2010; Hartling at al., 2013) . In particular, low agreement (with a k-value < 0.50 for eight of the nine questions) between two independent reviewers when using the NOS has been documented in some surveys. Tool's decision rules and some interpretative questions (e.g. whether exposed cohorts are somewhat or truly representative of the average exposed person in the community) may appear vague and difficult to use (Hartling et al, 2013) . Furthermore, NOS gives equal weight to each question, which sometimes may not be appropriate.

In combination with NOS we designed a questionnaire aimed at assessing crucial aspects of papers for data generalizability in clinical practice. This questionnaire was based on study reporting and may not reflect how the study was actually conducted.

Moreover, our analysis included early publications on COVID-19 and an improvement in quality of related studies has to be expected as the number of cases increases and better designed studies which take longer to design and conduct will be possibly published.

Accordingly, it is likely that over time research quality will improve.

It is our impression that many of the problems we have pointed to, were determined by a lack of coordination between different researcher groups and lack of multidisciplinary collaboration. A larger, possibly multi-country collaboration, and the involvement of researchers in different areas, including epidemiologists and biostatisticians, would increase the size of the studied populations, and allow refined analyses and higher quality results.

Such a collaboration would be of paramount importance when assessing the safety and immunogenicity of the vaccines against SARS-CoV-2.

The following issues should be carefully considered in future studies:

 Always consider a comparator group. The comparator could be either the general population, for population-based studies, or an independent group of patients with similar characteristics as cases, for hospital-based studies.

 Analyse separately different populations of patients (e.g. PsA separated from rheumatoid arthritis).

 Exclude from the analysis probable cases of COVID-19 (without a positive test) especially when making comparisons with the general population. These cases are typically not included in the data available from the general population, which only consider a COVID-19 patient the one who has been tested positive.

 Formally evaluate the sample size required to document a given incidence or prevalence rate, or an expected difference among study groups, looking at confidence intervals and not assuming negative results as proof of a lack of difference in underpowered studies.  Plan a priori sub analyses in high risk patient groups such as older patients or those with comorbidity.

 Even more importantly, to establish multicenter collaboration, prioritizing quality in data collection. A system to rapidly activate formal epidemiological studies and registries when confronted with global health crises should be considered, with an international study coordination and data sharing, as, for example, Psoprotect is (Mahil et al, 2021) .

Our study is not intended as criticism to the Journals or the Authors, who genuinely provided a service to the scientific community, but rather a reminder for readers to be careful when J o u r n a l P r e -p r o o f they read new COVID-19 papers. During a pandemic, one should be more cautious when incorporating evidence from new studies into the personal clinical decision-making.

Considering the currently published data, no definite statement can be made on the risk of COVID-19 among patients with psoriasis or psoriasis arthritis treated with biologics. At the moment, a cautious approach is still recommended. Better designed robust studies taking into account a suitable comparator, a proper sample size calculation and a confirmed ascertainment of incident cases are needed in order to reliably define the incidence and the outcome of COVID-19 in these patients.

The tremendous hunger for data by the public and medical community and the understandable desire of providing swift information should not, in the future, lower the quality of research. Articles that remained after the initial screening underwent a full-text review for inclusion consideration. For the quality assessment of studies case reports were excluded. The detailed search strategy is displayed in Figure 1 .

Data extraction and quality assessment J o u r n a l P r e -p r o o f All data were independently abstracted by two authors. For each of the selected study data on first author, study design, country, period of observation, sample size, presence of a control group, age and sex, number of SARS-COV-2 positive subjects, type of medications and COVID-19 clinical outcomes were collected.

The NOS (Newcastle-Ottawa Scale) (NOS) was used to estimate the internal validity of the included studies. The NOS is a tool developed jointly by the University of Newcastle (Australia) and the University of Ottawa (Canada) with the purpose of assessing the quality of nonrandomized studies to be used in systematic reviews. It consists in a "star system" in which a study is judged on three broad perspectives: the selection of the study groups; the comparability of the groups (of case and controls in case-control studies, of cohorts in cohort studies); and the assessment of the outcome (in case-control studies) or exposure (in cohort studies) (Wells et al, 2013) . For cross-sectional studies, a modified version of the NOS was adopted (Herzog et al, 2013) .

The NOS score for case-control and cohort studies ranges from 0 to 9, while the score for cross-sectional study ranges from 0 to 10, with higher scores indicating a better quality of the study. The NOS can also be normalized as a percentage score. NOS scores ≥75% are considered as high-quality studies (with a low risk of bias).

In the present work, four authors were involved in the rating process, with two blinded assessors rating in parallel each Pso and PsA study respectively. If there was any disagreement between assessor's rating, the discrepancy was further discussed. In case the disagreement could not be solved, the average of paired raters' scores was considered as the final result.

In addition, in order to provide a better appreciation of the internal validity and the external validity according to the Quality Criteria for Nontherapeutic Studies of the Agency for J o u r n a l P r e -p r o o f Healthcare Research and Quality (AHRQ), two independent raters assessed the following aspects of the studies: case definition, modality of COVID-19 assessment, evidence for selfselection of participants, existence of dropout/nonparticipants and their percentage and adequate sample size estimate (Shamliyan et al, 2011; Dekkers et al, 2010) . Any discrepancy in judgment by the two independent raters was resolved by discussion within the whole study group.

A detailed description of the generalizability criteria is presented in supplementary material (Table S5 ).

The present work seeks to evaluate the quality of the selected studies through the NOS and some additional points, in order to establish the robustness and reliability of the data published during the pandemic period. A further goal was to draw the attention to the necessity of having specific and shared criteria in studies conduction. The main outcomes of interest of the papers selected were the incidence and severity, in terms of hospitalization, intensive care unit (ICU) admission and mortality from SARS-COV-2 infection in patients with psoriasis treated by biologics.

Medications considered were, for PsA csDMARDs (conventional synthetic disease-modifying anti-rheumatic drugs), bDMARDs (biological disease-modifying antirheumatic drugs), tsDMARDs (targeted synthetic disease-modifying anti-rheumatic drugs); and for Pso Anti-TNF, Anti-IL23 Anti-IL12/IL23, Anti-IL17, Anti-IL23p19, conventional systemics, apremilast, dimetilfumarate.

Besides assessing the quality of the selected studies through the NOS and generalizability criteria we proposed recommendations for the conduction of future studies in this area. J o u r n a l P r e -p r o o f J o u r n a l P r e -p r o o f

Incidence of severe COVID-19 outcomes in psoriatic patients treated with systemic therapies during the pandemic: A Biobadaderm cohort analysis

Editorial Evaluation 416 and Peer Review During a Pandemic: How Journals Maintain Standards

Does therapy with biological drugs influence COVID-19 infection? Observational monocentric prevalence study on the clinical and epidemiological data of psoriatic patients treated with biological drugs or with topical drugs alone

in French patients with chronic inflammatory rheumatic diseases: Clinical features, risk factors and treatment adherence

Biologics increase the risk of SARS-CoV-2 infection and hospitalization, but not ICU admission and death: Real-life data from a large cohort during red-zone declaration

COVID-19 in patients with cutaneous immune-mediated diseases in The Netherlands: real-world observational data

Testing the Newcastle Ottawa Scale showed low reliability between individual reviewers

National registry for patients with inflammatory rheumatic diseases (IRD) infected with SARS-CoV-2 in Germany (ReCoVery): a valuable mean to gain rapid and reliable knowledge of the clinical course of SARS-CoV-2 infections in patients with IRD

Are healthcare workers' intentions to vaccinate related to their knowledge, beliefs and attitudes? A systematic review

Severe COVID-19 outcomes in patients with psoriasis

Against pandemic research exceptionalism

Factors associated with adverse COVID-19 outcomes in patients with psoriasis-insights from a global registry-based study

Incidence and case fatality rate of COVID-19 in patients with inflammatory articular diseases

Coronavirus disease 2019 (COVID-19) in autoimmune and inflammatory conditions: clinical characteristics of poor outcomes

More on Covid-19 in Immune-Mediated Inflammatory Diseases

Clinical outcomes of hospitalised patients with COVID-19 and chronic inflammatory and autoimmune rheumatic diseases: a multicentric matched cohort study

COVID-19: A Case for Inhibiting IL-17?

Lack of Evidence for an Increased Risk of Severe COVID-19 in Psoriasis Patients on Biologics: A Cohort Study from Northeast Italy

The impact of COVID-19 pandemic in a cohort of Italian psoriatic patients treated with biological therapies

Patient´s perspective: psychological burden of the COVID-19 pandemic in 146 psoriatic patients treated with biological drugs and small molecules in real clinical practice

Prevalence, or Risk Factors of Chronic Diseases: Pilot Study of New Checklists. Agency for Healthcare Research and Quality (US)

Critical evaluation of the Newcastle-Ottawa scale for the assessment of the quality of nonrandomized studies in meta-analyses

Effects of COVID-19 Scientific Pressure on Quality of Published Evidence. Presented at: Virtual European Society of Cardiology Congress

Psoriasis, biological drugs and Coronavirus Disease 2019: Real life experience of two Italian provinces

The risk of respiratory tract infections and symptoms in psoriasis patients treated with IL-17-pathway inhibiting biologics: A metaestimate of pivotal trials relevant to decision-making during the COVID-19 pandemic

NOS) for assessing the quality of nonrandomised studies in meta-analyses

Scientific quality of COVID-19

SARS CoV-2 publications in the highest impact medical journals during the early phase of the pandemic: A case control study

J o u r n a l P r e -p r o o f csDMARD: conventional synthetic disease-modifying anti-rheumatic drugs, bDMARD: biological disease-modifying antirheumatic drugs, tsDMARD targeted synthetic diseasemodifying anti-rheumatic drugs, NSAID: non-steroidal anti-inflammatory drug.J o u r n a l P r e -p r o o f 

Case definition Whenever it is not possible to consider data concerning a given disease entity because these data are combined with those of other conditions, the answer is "mixed-up."COVID-19 assessment It referrers to the means by which COVID-related conditions are assessed.Voluntary self-selection This applied when people are offered participation on a voluntary basis (e.g., be providing a link to a web questionnaire).

Usually samples from a target population are identified, contacted and recruited. If not all contacted people participate or participant people are lost to follow-up then there are nonparticipants or drop outs. If such a recruiting process is not clear then the response "unclear" is applied.% of dropout/nonparticipants It indicates the number without decimals.Sample size estimate For a "yes" answer a statement concerning statistical power or formal sample size calculation should be found in the paper.