key: cord-0846003-v3qm9k8k
authors: Paynter, Alex; Khan, Umer; Heltshe, Sonya L.; Goss, Christopher H.; Lechtzin, Noah; Hamblett, Nicole Mayer
title: A comparison of clinic and home spirometry as longtudinal outcomes in cystic fibrosis
date: 2021-08-31
journal: J Cyst Fibros
DOI: 10.1016/j.jcf.2021.08.013
sha: 729d9358c9c3cf36a5267e99b736ccded2b11ff1
doc_id: 846003
cord_uid: v3qm9k8k

BACKGROUND: The COVID-19 pandemic has accelerated the transition to telehealth, including the use of home spirometry in cystic fibrosis. Evaluating the accuracy and precision of longitudinal home spirometry is a requisite for telehealth-based research. This secondary analysis of a CF study (eICE) evaluates whether there are cross-sectional or longitudinal differences between home and clinic spirometry. METHODS: Participants age ≥14 years with ppFEV(1)>25 were recruited from 2011-2015, issued a home spirometer, and asked to complete spirometry efforts twice per week for one year. Clinic spirometry was collected at baseline and every three months. Cross-sectional differences between clinic spirometry and the closest home spirometry measurement were analyzed. Longitudinally, we apply 5 methods to analyze the precision of home spirometry, and differences between clinic vs. home data. RESULTS: Home spirometry is estimated to be 2.0 (95% CI: 0.3, 3.5) percentage points lower than clinic spirometry cross-sectionally. Longitudinally, the estimates of 12-month change in home spirometry varied by analysis method from -2.6 to -1.0 ppFEV(1)/ year, with precision markedly different. However, home spirometry change estimates were qualitatively similar to the clinic results: -3.0 ppFEV(1)/year (95% CI: -4.1, -1.9). CONCLUSIONS: To leverage the potential cost, feasibility and convenience of home spirometry, the differences with clinic spirometry must be acknowledged. Significantly lower ppFEV(1) in home devices shows that direct comparison to clinic spirometers may induce a spurious change from baseline, and additional variability in home devices impacts statistical power. The effect of coaching, setting, and equipment must be understood to use and improve home spirometry in CF.

The emergence of SARS-CoV-2 infection leading to the COVID-19 pandemic has created a situation where clinic-based spirometry measurements are difficult to obtain [ 1 , 2 ] . Further, the recent approval of elexacaftor/tezacaftor/ivacaftor for up to 90% of persons with CF [ 3 , 4 ] may make clinic-based research visits less desirable in a CF population with newfound milder disease. There is both an acute and long term need to evaluate the use of more flexibly derived spirometry endpoints. Acutely, during the COVID-19 pandemic, design changes were necessary to ensure participant safety and adhere to institutional restrictions on clinical research. As a result, some ongoing studies incorporated home spirometry at follow-up visits, with the intention to compare against clinic spirometry collected at baseline. The effect of this change on accuracy and precision of treatment effect is unknown, but it could impact interpretation and regulatory evaluation. For new studies in development, that are considering home spirometry over the whole study period, an assessment of sample size and power using home devices is warranted.

The Early Intervention in Cystic Fibrosis Exacerbation (eICE) study was a one-year randomized, controlled trial in adolescents and adults with CF aimed at determining whether protocolized remote monitoring including patient reported outcomes (PROs) and home spirometry would lead to more rapid identification of acute pulmonary exacerbations [5] . It was hypothesized that earlier iden-tification of pulmonary exacerbations would lead to benefits in pulmonary function. While enhanced monitoring was effective at more frequently identifying exacerbations, this did not translate into improved pulmonary function compared to the control group [6] . To date, the use of home or remote spirometry in CF clinical research has largely been as a clinical monitoring tool rather than a research endpoint [7] .

We performed a secondary analysis of the eICE home and clinic lung function measures to determine if there are systematic differences or decreases in precision comparing home and clinic spirometry in a longitudinal trial. Additionally, several common analytic methods are contrasted to suggest strategies for optimizing the use of home spirometry data in future CF research.

The study design for the eICE trial (NCT01104402) has been previously described [5] . From 2011 to 2015, 135 eligible study participants with age ≥14 years and forced expiratory volume in one second percent of predicted (ppFEV 1 ) > 25 were randomized into the early intervention arm from 13 participating centers. Participants in the early intervention arm were issued an AM2 + R Lung Function Monitor (ERT, Inc.) [8] to be used twice weekly for 12 months. This analysis includes the 133 participants with at least one home spirometry measurement recorded. The device was designed and tested to meet American Thoracic Society standards [9] . Participants received an in-person demonstration of the home spirometry device from a research coordinator, technician, or other personnel (not necessarily the same personnel who collected clinic spirometry). Written instructions were also provided. No maintenance of home spirometry technique after the initial demonstration was attempted. Spirometry was to be measured in a seated position, after taking inhaled mediations and performing airway clearance techniques. Participants performed clinic spirometry on the standard of care schedule, once every 3 months over the study duration and in accordance with the 2005 ATS/ERS guidelines [9] . To encourage adherence, if participants had not uploaded a home spirometry measurement in 5 days they were contacted by the care center. Participation in the optional text message reminder system was low. While nose clips were used in the clinic per standard research practice, they were not used in the home setting. For each pulmonary function test (clinic or home), the highest value from a session was used in analysis. The 2012 Global Lung Initiative equations were used to calculate ppFEV 1 , which differs from the original eICE publication [6] .

Cross-sectional analyses evaluate pairwise differences (homeclinic) on nearby home and clinic observations for each participant. The nearest neighbor (NN) method pairs each clinic observation to a single home spirometry observation, specifically the nearest neighbor in time (limited to observations within 7 days). The windowed mean (WM) method pairs each clinic observation to the mean of all home spirometry measures within a ±7d window. Sensitivity analyses with greater window sizes are included for WM. Both methods utilize mixed models with a random intercept to account for the correlation of observations within each participant.

Longitudinal analyses ( Fig. 1 ) evaluate the change from baseline and are applied independently to home and clinic data. NN and WM use the matching described above to select home spirometry observations, with change from baseline at each visit as the outcome, rather than the difference between home and clinic. The individual participant regressions (IPR) method first calculates individual linear regressions with time for each participant, then the slopes are analyzed across participants as derived variables. IPR is limited to subjects with at least 175 days of follow-up. The fourth method is mixed model, including a random subject intercept, and a linear relationship allowed for time (MMLT). The last method is a mixed model with a spline for time (MMST), specifically a natural cubic spline [11] . A quick reference for the methods is in Table 1 , longitudinal methods are diagrammed in Fig. 1 , and additional details on each method are provided in the supplement.

Regarding the selection of longitudinal methods, NN and MMLT are among the most common analytical approaches in cystic fibrosis trials. NN amounts to a paired t test at each visit, and MMLT is one of many models called a "mixed model for repeated measures". WM and IPR have been proposed elsewhere as strategies which may be especially suited to home spirometry [10] . MMST adds some flexibility to MMLT, the primary motivation is that it may be suitable in a broader range of CF studies.

Statistical significance is defined by two-sided testing at the 0.05 level with no adjustment for multiple comparisons. Throughout we indicate 95% confidence intervals in parentheses, and all confidence intervals are derived using a nonparametric bootstrap with the percentile method. For any mixed model the bootstrap is resampled on the cluster (participant) level [12] , and a note on the impact of this choice is included in the supplement. The R programming language [13] was used for all analyses, and a code tutorial is available online [14] .

The 133-participant cohort had a mean age of 26.5 years (SD = 11.5), and their mean baseline clinic ppFEV 1 was 78.9% (SD = 22.0). 55 were F508del homozygous (41%) while 64 were F508del heterozygous (47%). Supplemental Table 1 contains additional baseline demographics as does the primary paper for the trial [6] . The median number of home spirometry measures was 56 per person. Adherence to the twice weekly home spirometry protocol decreased over the study period, with the number of home spirometry observations per subject-week being 1.81 at baseline, A mixed model with a participant-level random intercept is fit using a natural spline for time.

Yes * These analyses operate on a per-visit basis and thus the longitudinal analysis is really a change from baseline. Additionally, a maximum of one observation per clinic visit can be incorporated by them -or one window per visit for WM. visit and 98 at the 39 week visit. A more complete description follow-up as it relates to the windows is available in Supplemental Table 2 . While adherence decreased over the study, most subjects have some long-term home spirometry follow-up. Fig. 2 A displays the cross-sectional differences in ppFEV 1 , using all available data from the one-year study. Regardless of the method of estimation, a statistically significant difference exists between home and clinic spirometry. Using the NN method, the average within-participant difference between clinic spirometry and the nearest neighbor home spirometry observation is -2.01 (95% CI,-3.5, -0.3) ppFEV 1 , indicating that home spirometry is systematically less than clinic spirometry. Using WM, the magnitude of difference is greater with larger windows, down to -2.43 ppFEV 1 with a 28d windowed mean. Fig. 2 B shows the results of the NN analysis separated by visit to assess whether differences between home and clinic spirometry attenuate over time. The difference estimates remain negative at all visits, and there is insufficient evidence to show a systematic training effect over time. However, this analysis includes only those with all five time points (n = 44) to eliminate censoring bias, so the sample available is small.

A difference versus mean plot (Bland-Altman, Fig. 3 ) using NN matching shows that some of the differences exceed 30 ppFEV 1 , and in the majority of outliers the home observation is greater than the clinic observation. Therefore, any post-hoc outlier exclusions would only enlarge the average cross-sectional difference. There was no significant association between spirometry differences and age or sex (Supplemental Figure 4) . However, most of the extreme differences occurred in adolescents and young adults.

The longitudinal results focus on the estimated change over time, rather than cross sectional difference. Estimates derived in- 

There are important unanswered questions as to whether changes in pulmonary function captured via home spirometry measures are different from clinic spirometry, whether this difference changes over time, and if home spirometry results in a loss of precision compared to clinic spirometry. Our study begins to address these questions through a rigorous retrospective analysis of data from the eICE study. We have demonstrated a cross sectional difference between home and clinic spirometry in these data, and shown that the method of analysis can have a dramatic impact on the longitudinal analysis of home spirometry.

The existence of a cross sectional difference in ppFEV 1 between home and clinic spirometry has been previously reported in patients with chronic diseases [ 15 , 16 ] . We have expanded on this work by confirming that a cross-sectional difference exists in a CF study population utilizing home spirometers. Additionally, we have shown that this difference does not resolve quickly over the course of a one-year study. A windowed mean was only modestly effective in reducing the variability of this difference, and it did not help at all with the bias. Larger windows exacerbated the crosssectional bias. While other work generally supports the conclusion that home spirometry tends to be lower than clinic spirometry, the home versus clinic variability is sometimes lower than our results indicate [15] . Home spirometry was used as a monitoring tool in eICE, not a planned research outcome, and we cannot over-emphasize how important this context is in interpreting the results. Higher variability is likely a consequence of the study design. Specific contributors to lower variability in other work may include equipment (especially nose clips), data exclusions, coaching, testing of home devices in a clinic setting, or an unmea-sured factor. Even two clinic spirometers used in the same setting will vary [17] , so a certain portion of the difference is likely irreducible. Future work designed to parse the impact of each contributor will aid recommendations for counteracting cross-sectional bias.

Longitudinal evaluations of home spirometry have been conducted in related diseases and spirometric outcomes. Studies of forced vital capacity in idiopathic pulmonary fibrosis have pointed out that longitudinal outcomes may differ even if home and clinic spirometry are cross sectionally similar [ 18 , 19 ] . One of these studies concluded that the greater variability of home spirometry would require greater sample sizes to maintain power [19] , while the other calculated that if the frequency of home spirometry is sufficiently high, that may yield superior longitudinal precision [18] . In a recent phase 3 randomized, placebo-controlled interventional trial in Duchenne muscular dystrophy (DMD) whereby both clinic and home spirometry were captured longitudinally over a year-long study, a retrospective study was performed to evaluate the comparability of treatment effect estimates derived from home vs. clinic-based peak expiratory flow [10] . Similar treatment effect treatment estimates were obtainable from remote spirometry and clinic spirometry. However, the analytical approach which produced treatment effect estimates closest to those obtained from the clinic spirometer suffered from a notable loss of precision. Our longitudinal analysis confirms the finding that home spirometry had lower longitudinal precision, and thus would require larger sample sizes to maintain power.

In our comparison of longitudinal analysis methods, we found that methods which model home spirometry over time (IPR, MMLT, or MMST) have superior performance compared to methods which analyze on a per-visit basis (NN or WM). This conclusion follows from the qualitatively similar longitudinal estimates using home data with IPR, MMLT or MMST, as compared to clinic data, and the superior precision compared to the nearest neighbor or windowed mean methods. Visit-based analyses are still pervasive for clinic spirometry, but the unstructured, frequent measurements in home spirometry make the difference between methods sharper. This finding agrees with previous work [18] . The magnitude of 12-month change in eICE was greater than in a typical CF study, but we would expect our findings to generalize to cohorts with a more modest ppFEV 1 decline. While the IPR method is easily implemented and had good performance here, there are well known theoretical advantages to mixed models which properly account for within-subject correlation. As a result, we recommend the MMLT method as a good starting point for future analyses of home spirometry.

Home spirometry, particularly if unstructured observation timing or differential adherence exists, leads to a vastly different cluster size (observations per participant). Our use of cluster-level resampling is motivated by this observation, and we found that confidence interval method substantially impacts the results using home data but not clinic data (see supplement). For studies in this situation, we recommend a similar process to obtain confidence intervals. Home spirometry analyses which have a more rigid structure and high adherence, for example if the data collection was capped at once per week and adherence was high, may not need this additional complexity. In other words, it is more likely that the cluster size will vary in a home spirometry study design, but this is not a fundamental property to using home devices. Large outliers also contribute to the need for cluster-level bootstrap methods, which are likely reducible with more stringent real-time quality control.

While IPR and MMLT increased the precision of our longitudinal estimates, the imposition of a linear trend with time may not be universally suitable. This would be especially true for any study where participants experience an acute benefit from therapy, such as the recent trials of cystic fibrosis transmembrane conductance regulator (CFTR) modulators. Allowing for a nonlinear relationship with time has been shown to substantially impact studies of FEV 1 in CF [20] . For cases when a nonlinear trend is likely, the MMST method, or any method which allows a nonlinear trend to be fit, will offer a more nuanced result, and decrease bias.

There are important challenges in home spirometry we have not addressed in this paper. COVID-19 has left many studies with a mix of home and clinic spirometry data, and we have not evaluated methods which attempt to model both concurrently. A simulation study, which would allow more precise notions of accuracy, precision, and coverage, may be appropriate to evaluate candidate methods.

There are several limitations with the analysis presented. First, adherence to home spirometry was low overall and it was lower than clinic visit adherence, which may have introduced bias related to missing data. Second, as noted above adherence waned over time making conclusions about longer term follow-up more challenging. Third, both technology and potentially the clinical stability of lung function has changed over time since this original study was conducted.

Home spirometry offers great promise in finding a safe, convenient, and cost-effective alternative to measuring pulmonary function in CF in the COVID-19 era and beyond. We provide cautionary results suggesting that assuming home and clinic spirometry are interchangeable may yield unsatisfying or misleading conclusions. Nonetheless, with appropriate administrative and analytical adaptations, home spirometry can become a useful tool in CF research.

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. 

NMH was supported by the Cystic Fibrosis Foundation (CFF) and National Institutes of Health (NIH) grant UL1 TR002319 . CHG was supported by grants from the Cystic Fibrosis Foundation , the NIH ( UM1 HL119073 , P30 DK089507 , U01 HL114589 , UL1 TR0 0 0423 ) and the FDA ( R01 FD0 03704 , R01 FD0 06 84 8 ). SLH was supported by the Cystic Fibrosis Foundation (CFF) and National Institutes of Health (NIH) grant P30 DK 089507 . UK and AP are supported by grants from the Cystic Fibrosis Foundation (CFF).

Supplementary material associated with this article can be found, in the online version, at doi:10.1016/j.jcf.2021.08.013 .

SARS-CoV-2 disrupts clinical research: the role of a rare disease-specific trial network

Statistical considerations for clinical trials during the COVID-19 public health emergency guidance for Industry

Elexacaftor-tezacaftor-ivacaftor for cystic fibrosis with a single Phe508del allele

Efficacy and safety of the elexacaftor plus tezacaftor plus ivacaftor combination regimen in people with cystic fibrosis homozygous for the F508del mutation: a double-blind, randomised, phase 3 trial

Rationale and design of a randomized trial of home electronic symptom and lung function monitoring to detect cystic fibrosis pulmonary exacerbations: the early intervention

eICE study team. Home monitoring of patients with cystic fibrosis to identify and treat acute pulmonary exacerbations. eICE study results

Advancing outcome measures for the new era of drug development in cystic fibrosis

New telemetric system for daily pulmonary function surveillance of lung transplant recipients

Standardisation of spirometry

Home-based monitoring of pulmonary function in patients with duchenne muscular dystroph

The elements of statistical learning: data mining, inference, and prediction

Handbook of multilevel analysis

Core Team R: A language and environment for statistical computing, Vienna, Austria: R Foundation for Statistical Computing

Validation of the portable Bluetooth® Air Next spirometer in patients with different respiratory diseases

Technical validity and usability of a novel smartphone-connected spirometry device for pediatric patients with asthma and cystic fibrosis

Technical and functional assessment of 10 office spirometers: a multicenter comparative study

Home monitoring improves endpoint efficiency in idiopathic pulmonary fibrosis

INMARK trial investigators. Home spirometry in patients with idiopathic pulmonary fibrosis: data from the INMARK trial

A semiparametric approach to estimate rapid lung function decline in cystic fibrosis