key: cord-333078-8cto831y
authors: Kruizinga, Matthijs D.; Essers, Esmée; Stuurman, F. E.; Zhuparris, Ahnjili; van Eik, Nellie; Janssens, Hettie M.; Groothuis, Iris; Sprij, Arwen J.; Nuijsink, Marianne; Cohen, Adam F.; Driessen, Gertjan J. A.
title: Technical validity and usability of a novel smartphone‐connected spirometry device for pediatric patients with asthma and cystic fibrosis
date: 2020-07-08
journal: Pediatr Pulmonol
DOI: 10.1002/ppul.24932
sha: 
doc_id: 333078
cord_uid: 8cto831y

BACKGROUND: Diagnosis and follow‐up of respiratory diseases traditionally rely on pulmonary function tests (PFTs), which are currently performed in hospitals and require trained personnel. Smartphone‐connected spirometers, like the Air Next spirometer, have been developed to aid in the home monitoring of patients with pulmonary disease. The aim of this study was to investigate the technical validity and usability of the Air Next spirometer in pediatric patients. METHODS: Device variability was tested with a calibrated syringe. About 90 subjects, aged 6 to 16, were included in a prospective cohort study. Fifty‐eight subjects performed conventional spirometry and subsequent Air Next spirometry. The bias and the limits of agreement between the measurements were calculated. Furthermore, subjects used the device for 28 days at home and completed a subject‐satisfaction questionnaire at the end of the study period. RESULTS: Interdevice variability was 2.8% and intradevice variability was 0.9%. The average difference between the Air Next and conventional spirometry was 40 mL for forced expiratory volume in 1 second (FEV1) and 3 mL for forced vital capacity (FVC). The limits of agreement were −270 mL and +352 mL for FEV1 and −403 mL and +397 mL for FVC. About 45% of FEV1 measurements and 41% of FVC measurements at home were acceptable and reproducible according to American Thoracic Society/European Respiratory Society criteria. Parents scored difficulty, usefulness, and reliability of the device 1.9, 3.5, and 3.8 out of 5, respectively. CONCLUSION: The Air Next device shows validity for the measurement of FEV1 and FVC in a pediatric patient population.

Diagnosis and longitudinal follow-up of pulmonary diseases have relied on pulmonary function tests (PFTs) since the nineteenth century. 1 Traditionally conducted in the clinic, spirometry can be a difficult technique, and the accuracy and repeatability depend on many factors such as equipment, patient effort, and supervision and encouragement of a technician. Nevertheless, a single PFT is no more than a snapshot of disease activity, and is unable to capture the variability of symptoms in chronic pulmonary disease.

Longitudinal data on a regular basis regarding pulmonary health could be very valuable for patients, clinicians, and clinical researchers, and this could be obtained by performing PFTs at the patients' home.

An increase in readily available objective longitudinal data could be particularly useful in pediatrics, as children often find it difficult to perceive and express the severity of their symptoms. 2, 3 Researchers have investigated the clinical value of home-based measurements of several devices for pediatric asthma and cystic fibrosis (CF). While pulmonary outcomes were correlated to disease activity, 4 the devices appeared to offer little benefit for clinical practice in terms of reduced admission rates, better disease control, or slower decline in pulmonary function. [5] [6] [7] Since then, improvements in technology have allowed for the development of devices for measurement of complete flow-volume curves at relatively low cost.

An example is the Air Next spirometer, a Bluetooth connected device, allowing patients to perform spirometry tests with a smartphone. Use of the device has been reported in adult patients, but not yet in the pediatric population. 8, 9 Before implementation in pediatric clinical care or clinical trials, a comprehensive technical validation of the device must be performed, consisting of the assessment of intraand interdevice variability, comparison with conventional spirometry, as well as the assessment of usability for pediatric patients.

The aim of this study is to determine the agreement between the Air Next spirometer and conventional spirometry and to evaluate the usability of the device for children and parents when used at home. 

This analysis was part of a study investigating a novel homemonitoring platform (CHDR MORE) in pediatrics. During this study, pediatric patients with controlled asthma (n = 30), uncontrolled asthma (n = 30), and CF (n = 30) were recruited from the outpatient clinic of the hospitals. All children were aged between 6 and 16 years.

Asthma control was defined using the Global Initiative for Asthma criteria and Asthma Control Questionnaire (cutoff > 1.5 points). 10, 11 Children and parents were given a 10-minute training and practice session and were asked to perform PFTs once daily with the mobile device for a duration of 28 days. When logistically feasible, children visited the hospital to perform a conventional spirometry test at the outpatient clinic at the beginning or end of the study period and performed an Air Next spirometry test during the same visit. The sequence of tests was chosen based on preference for each patient. 

The Air Next device cannot be manually calibrated. We used a calibrated syringe (Viasys, Conshohocken, PA) with a capacity of 2994 mL to evaluate accuracy and the inter-and intradevice variability. The syringe was used to push the complete capacity through an Air Next device 20 times per device on 20 devices with a single turbine. In addition, the syringe was used on 25 different turbines with a single Air Next device.

ATS and ERS acceptability guidelines were used to judge and grade PFT quality (grade A-F from best to worst). 13 Spirometry maneuvers were acceptable if the start was rapid and without hesitation, the 2464 | course of the expiratory maneuver was continuous, without any artefacts or evidence of coughing in the first second and if the end of the maneuver did not show early or abrupt interruption. The difference between the best two acceptable FVC and FEV1 should have been less than 150 mL. At least three maneuvers were performed per spirometry session. When it was difficult to obtain reproducible maneuvers during supervised measurements, a maximum of 10 maneuvers per patient were performed and the usable maneuvers were used. For home use, subjects were instructed to perform three maneuvers per session and were able to perform two additional measurements when appropriate (for example, mistiming of the forced exhalation or application errors). Subjects were not asked to self-grade repeatability during the study period.

At the end of the study period, a questionnaire regarding user experience was completed. Parents and participants were asked to give their opinion about the reliability of the device, the difficulty of using the device, and whether they found the use of the device to be useful or tedious on a 5-point Likert scale.

Baseline characteristics were summarized. Inter-, intra-, and turbine variability were calculated and expressed as a coefficient of variability (CV). Concordance between Air Next spirometry and conventional spirometry was assessed using the methods described by Altman and Bland. 14 The mean differences between methods and the 95% limits of agreement were calculated for FEV1, FVC, PEF, and FEV1/ FVC ratio. For FEV1 and FVC, acceptable bias was no more than 100 mL.

For PEF and FEV1/FVC ratio, the acceptable average bias was 300 mL/s and 10%, respectively. 13, 15 Pearson correlation coefficients between the two methods were calculated. Spirometry measurements at home were graded for quality and the number of maneuvers assigned to each grade were summarized descriptively. A mean grade per subject was calculated.

The average mean grades of the three study groups were compared via a one-way analysis of variance test and pairs were compared with Tukey's range test to adjust for multiple comparisons. Usability was evaluated by analyzing the end-of-study questionnaire completed by subjects and their parents. R version 3.5.1 was used for statistical analysis and visualization.

Promasys software (OmniComm, Lauderdale, FL) was used for data management.

A total of 90 subjects were included in the main study. The average age was 10 years (range, 6-15). Subjects had performed an average of 12 (SD 11) hospital-based PFTs before the study. Other baseline characteristics are displayed in Table 1 .

Of 400 measurements in 20 devices, the average bias from the calibrated 2994 mL was −40 mL (range, −124 to 56 mL). The average intradevice CV was 0.9% (range, 0.6%-1.2%). Furthermore, the average interdevice CV was 2.8%. Average turbine bias was −70 mL and turbine CV was 1.8%. About 4% of measurements with the calibrated syringe exceeded the 3% accuracy threshold advised by ATS standards.

Fifty-eight subjects were able to perform hospital and Air Next PFTs subsequently. When comparing output between the two methods, there was one extreme outlier, most likely due to a technical defect resulting in a blockage of the outflow of the Air Next turbine, which was excluded from the statistical analysis. Figure 1 shows the limits of agreement and correlation between the Air Next and conventional spirometry of the several parameters. For FEV1, the average bias was 40 mL and the 95% limits of agreement were −270 and +352 mL. there was still a good correlation between the two methods for both PEF (R = .93, P < .001)) and FEV1/FVC ratio (R = .91, P < .001). There was no proportional bias for any of the parameters. There was a correlation (R = −.33, P = .01 for FEV1 and R = −.26, P = .05 for FVC) between the absolute difference in FEV1 and FVC (expressed in % of predicted FEV1 and FVC) and age ( Figure S1 ), but not between the absolute difference and previous spirometry experience, expressed as the amount of PFTs performed in the past ( Figure S2 ). There was no statistically significant difference in absolute bias for FEV1 between the three groups (P = .28; Figure S3 ). When the absolute difference between the two methods was expressed as a percentage of the predicted FEV1 and FVC, the mean bias was 6.3% (SD 5%) of predicted FEV1 and 6.7% (SD 5.7%) of predicted FVC. The bias of FEV1 of subjects who performed the comparison at the end of the study period was slightly higher (3% of predicted, P = .009) compared to subjects who performed the comparison at the beginning of the study period ( Figure S4 ).

A total of 2047 spirometry measurements were performed with the Air Next device during the course of the study, resulting in an average compliance of 78%. The curves of 1821 sessions were available for analysis. When graded according to the ATS/ERS criteria, 45% of the FEV1 measurements were considered acceptable and reproducible, as well as 41% of the FVC measurements. A significant number of sessions were grade E, meaning they did not produce more than one acceptable maneuver or that the reproducibility was too low. About 2% of measurements were neither acceptable nor usable for both FEV1 and FVC. Summarized grades are listed in Figure 2A ,B. There was a statistically significant difference on average grade between CF patients and patients with uncontrolled asthma (FEV1, P = .02; Figure 2C and FVC, P = .03; Figure 2D ). Age and average grade were not correlated ( Figure S5 ).

Day-to-day CV of acceptable trials (grade A-C) was 9.0% (SD 5.7%) for FEV1 and 7.7% (SD 5.4%) for FVC.

Sixty-nine (77%) subjects completed the end-of-study questionnaire.

In general, parents found the use of the spirometry device to be acceptable. When asked to score their agreement with the statement "I found the use of the spirometer to be tedious," the average score was 1.8 out of 5 (SD 1.1). Furthermore, parents scored the difficulty 1.9 out of 5 (SD 1.2), usefulness 3.5 out of 5 (SD 0.9) and the perceived reliability 3.3 out of 5 (SD 1.0). Summarized results are displayed in Figure S6 .

The current study investigates the technical validity and user experience of the Air Next spirometer for pediatric patients. Air

Next spirometer output was compared with the gold standard:

conventional spirometry in the clinic. Subjects and their parents also completed a questionnaire regarding the usability of the device. F I G U R E 2 ERS/ATS grades for measurements performed at home. All spirometry sessions were graded according to ATS/ERS guidelines for FEV1 and FVC separately. Grade A-E represent sessions with acceptable maneuvers but with varying repeatability. Grade U includes session with usable but not with acceptable maneuvers and grade F is reserved for session without acceptable or usable maneuvers. A, Proportion of spirometry sessions that were awarded each grade for FEV1. B, Proportion of spirometry sessions that were awarded each grade for FVC. C, Boxplot of average FEV1 grade per study group. Dots represent individual averages. There was a statistically significant difference between the CF and uncontrolled asthma group (P = .02). D, Boxplot of average FVC grade per study group. Dots represent individual averages. There was a statistically significant difference between the CF and uncontrolled asthma group (P = .03 This study has some limitations, one of which is that not all of the participants could be included in the validation group. This is mainly due to logistical reasons and the fact that the comparison was part of a secondary analysis of a clinical study. However, there were no large differences in baseline characteristics between the complete cohort and the validation cohort ( Table 1 ). The nonrandomized order of tests may have influenced the results through spirometry-induced bronchoconstriction. 25 However, we did not diagnose this condition in any of the included subjects. The curves of 226 spirometry sessions were unavailable for review due to application connectivity errors.

However, this issue occurred at random and, therefore, did not impact our overall conclusions. Although we found no correlation between the absolute bias and previous spirometry experience when comparing conventional spirometry to the Air Next, the proportion of highly the Air Next will be performed to determine the objectivity and reproducibility of longitudinal unsupervised measurements.

The Air Next spirometer is technically valid for the measurement of 

Contributions to vital statistics, obtained by means of a pneumatic apparatus for valuing the respiratory powers with relation to health

Poor perception of airway obstruction in children with asthma

Perception of pulmonary function in children with asthma and cystic fibrosis

Home spirometry and asthma severity in children

Home telemonitoring (forced expiratory volume in 1 s) in children with severe asthma does not reduce exacerbations

Impact of home spirometry on medication adherence among adolescents with cystic fibrosis

Home monitoring of patients with cystic fibrosis to identify and treat acute pulmonary exacerbations eICE study results

Validation of the portable Air-Smart spirometer

The utility of hand-held mobile spirometer technology in a resource-constrained setting

Identifying "well-controlled" and "not well-controlled" asthma using the Asthma Control Questionnaire

Asthma control questionnaire in children: validation, measurement properties, interpretation

Report of the Global Lung Function Initiative (GLI), ERS Task Force to establish improved lung function reference values, includng supplement

Standardization of spirometry 2019 update an official American Thoracic Society and European Respiratory Society technical statement

Measurement in medicine: the analysis of method comparison studies

Primary care spirometry

Asthma outcomes: pulmonary physiology

Validation of the portable Air-Smart spirometer

Technical and functional assessment of 10 office spirometers: a multicenter comparative study

Assessment of accuracy and applicability of a new electronic peak flow meter and asthma monitor

Comparison of a handheld turbine spirometer to conventional spirometry in children with cystic fibrosis

Recommendations for a standardized pulmonary function report. An official American Thoracic Society technical statement

Determinants of correct inhalation technique in children attending a hospital-based asthma clinic

A virtual asthma clinic for children: fewer routine outpatient visits, same asthma control

The future of clinical trial design: the transition from hard endpoints to value-based endpoints

Spirometry-induced bronchial obstruction

Technical validity and usability of a novel smartphoneconnected spirometry device for pediatric patients with asthma and cystic fibrosis

The authors would like to thank Mohamed Chouchouh el Khattabi for performing the variability assessments, the pulmonary function technicians for performing the conventional pulmonary function tests, and the clinical trial assistants and other support staff at the KRUIZINGA ET AL.

Centre for Human Drug Research. The study was funded by the Centre for Human Drug Research. This was an investigator-initiated study by the Centre for Human Drug Research, an independent clinical research foundation in Leiden, the Netherlands.

The authors declare that there are no conflict of interests.

MK conducted and designed the study, analyzed the data, and wrote the manuscript; EE conducted the study, analyzed the data, and reviewed the manuscript; NE performed measurements and reviewed the manuscript; HJ, IG, AS, and MN recruited patients and reviewed the manuscript; AZ supported data analysis; FS and AC designed the study and reviewed the manuscript; and GJD designed the study, supervised study conduct, and reviewed the manuscript.

All data is available from the corresponding author upon reasonable request.

http://orcid.org/0000-0002-2054-2187