key: cord-0875487-fe1v3hs1
authors: Browne, Sara H.; Bernstein, Mike; Pan, Samuel C.; Garcia, Jonathan Gonzalez; Easson, Craig A.; Huang, Chung-Che; Vaida, Florin
title: Maxim Integrated Smartphone Sensor with App Meets FDA/ISO Standards for Clinical Pulse Oximetry and can be Reliably Utilized by a Wide Range of Patients.
date: 2020-09-11
journal: Chest
DOI: 10.1016/j.chest.2020.08.2104
sha: a4a2cad2e868aa95cbf361380cd89d7985073fcf
doc_id: 875487
cord_uid: fe1v3hs1

Background Millions of smartphones contain a photoplethysmography (PPG) bio-sensor (Maxim Integrated, San Jose CA) that accurately measures pulse oximetry. No clinical use of these embedded sensors is currently being made, despite the relevance of remote clinical pulse oximetry to the management of chronic cardiopulmonary disease, and the triage, initial management and remote monitoring of persons effected by respiratory viral pandemics, such as SARS-CoV-2 or Influenza. To be used for clinical pulse oximetry the embedded PPG system must be paired with an App and meet FDA and ISO requirements. Research Question We evaluated whether this smartphone sensor with App met FDA/ISO requirements and how measurements obtained using this system compared to hospital reference devices across a wide range of persons. Study Design and Methods We performed laboratory testing addressing ISO and FDA requirements in ten participants using the smartphone sensor with App. Subsequently, we performed an open label clinical study on 320 participants with widely varying characteristics, to compare accuracy and precision of readings obtained by patients, to hospital reference devices using a rigorous statistical methodology. Results ‘Breathe Down’ testing in the laboratory showed that the total Root Mean Square Deviation (RSMD) of SpO2 measurement was 2.02%, meeting FDA/ISO standards. Clinical comparison of the smartphone sensor with App versus hospital reference devices determined SpO2 and heart rate (HR) accuracy was 0.48 % points (CI 0.38 to 0.58; p<0.001) and 0.73 bpm (CI 0.33 to 1.14; p<0.001) respectively; with SpO2 and HR precision 1.25 versus reference 0.95 points (p< 0.001) and 5.99 versus reference 3.80 bpm (p<0.001), respectively. These small differences were similar to the variation found between two FDA approved reference instruments for SpO2: accuracy 0.52 points (CI 0.41 to 0.64; p<0.001) and precision 1.01 versus 0.86 (p<0.001). Interpretation Our findings support the application for full FDA/ISO approval of the smartphone sensor with App tested for use in clinical pulse oximetry. Given the immense and immediate practical medical importance of remote intermittent clinical pulse oximetry to both chronic disease management and the global ability to respond to respiratory viral pandemics, the smartphone sensor with APP should be prioritized and fast tracked for FDA/ ISO approval to allow clinical use.

Millions of smartphones contain photoplethysmography (PPG) bio-sensors with Apps that accurately measure heart rate (HR) and blood oxygen saturation (SpO 2 ). 1,2 High grade PPG biosensors (Maxim Integrated, San Jose, CA), are currently integrated into 15% of android smartphones worldwide, totaling over 300 million smartphones. 3 The PPG sensor measures the distension of the arteries and arterioles in the subcutaneous tissue due to blood flow with each cardiac cycle. As blood flows through the vessels, the pulse pressure is detected by illuminating the skin with the light from two light-emitting diodes. The amount of light either transmitted, absorbed, or reflected to a photodiode is measured. A signal processing App, containing an J o u r n a l P r e -p r o o f 4 algorithm then detects and interprets the PPG signal to determine SpO2 and HR values. Finally, additional software within the App then accesses and displays individual bio-sensor readings on the smartphone screen.

Direct use of this technology for remote clinical pulse oximetry is highly relevant to the expanding integration of digital technologies into clinical care models. The utility of smartphone embedded sensors and Apps for intermittent clinical pulse oximetry is broad, potentially supporting the effective management of a wide range of chronic cardiopulmonary disease, including congestive heart failure (CHF), chronic obstructive pulmonary disease (COPD), or acute disorders such as pneumonia, and post-operative recovery. 4, 5, 6 Smartphone sensors with Apps for clinical pulse oximetry would allow patients to gather and track their own data to inform out-patient clinic or telemedicine visits, and increase communication with healthcare providers, which may lead to earlier out-patient interventions potentially reducing both morbidity and hospitalization risks. Smartphone sensors with Apps for clinical pulse oximetry may be of considerable importance in low-and middle-income countries (LMICs) with less developed healthcare infrastructure enabling management support of infections prevalent in these settings such as AIDS defining pneumonias like pneumocystis or cryptococcosis, tuberculosis, as well as chronic cardiopulmonary diseases. 7, 8, 9, 10, 11, 12, 13 Further, during events of pandemic respiratory viral spread, such as the current SARS CoV-2 or Influenza, remote clinical pulse oximetry may support not only triage, but also the initial management of symptomatic adults. 14,15 SpO 2 correlates with lung injury 15 and in conjunction with clinical exam may be used to differentiate those that require close monitoring and hospital admission, from those with milder disease. 6 In circumstances of home quarantine, remote clinical pulse oximetry could allow patients to follow their symptoms and include J o u r n a l P r e -p r o o f 5 objective report of SpO 2 and HR. Smartphone sensors with Apps could allow measurements collected, with patient permissions, to be connected to regional and national data hubs allowing large scale regional data analysis in near-real time. This could allow estimates of resource distribution, such as hospital bed numbers needed, and could provide means for medical centers to contact and communicate with patients in home quarantine. The availability of smartphone sensors with Apps for clinical pulse oximetry under such circumstances could be critical to lowand middle-income countries (LMICs), where medical infrastructure is easily overwhelmed.

Despite the existence of such a large number of smartphones embedded with these sensors with Apps, no clinical use is being made of these. Clinical pulse oximetry requirements by FDA /ISO for PPG sensors is < 3.5% root mean square deviation (RMSD) of SpO 2 value.

Herein we present data obtained using a smartphone sensor with App (Maxim Integrated, San Jose, CA) 16, 17 and report laboratory testing addressing FDA/ ISO requirements.

Subsequently, we report findings from an open label clinical study across a varied population, using a protocol that obtains the internal measurement error for smartphone sensor with App readings and hospital standard of care (SOC) reference instruments, and allows comparison of measurement sets to analyze both accuracy and precision. Data obtained using the smartphone sensor with App across a varied population provides data on how accurately it can be used by the general population. and Asian (4) racial/ethnic origins. All volunteers provided written informed consent. In each test volunteers placed their index finger over the smartphone sensor system, and a strap was placed over their hand to maintain it in place. The volunteer breathed a mixture of gases with reduced O 2 compared to room air. By manipulating this mixture, the volunteer's SpO 2 was decreased in steps. At each step, the measurements were allowed to reach a stable plateau, which was held for at least 45 seconds, before proceeding to the next step. Only the last 30 seconds of each plateau were used in the data analysis. Data was collected at SpO 2 levels near 100% to around 70%. SpO 2 values were recorded continuously, an FDA/ISO approved Massimo Radical reference device (Masimo, Irvine, CA) recorded SpO 2 simultaneously.

HR measurements data was taken using a ProSim8 Fluke Instruments (Fluke, Everett, WA).

Laboratory Testing Data Analysis: Each plateau was checked for quality of data and was rejected if: the plateau was not stable within +/-2% SpO 2 ; either the Masimo reference or the test device did not report a value; motion of the volunteer was observed; or perfusion, defined as optical modulation of the transmitted light, was less than 0.5%. Data were analyzed following Participants were adults over 18 years of age recruited from UCSD Healthcare or Clinical Trials Units (CTU) and provided written informed consent. Initially participants were recruited from out-patient settings (n=250). For each participant, four pairs of measurements (8 data points total) were taken in succession, with no break between sets, in the order indicated in Table 1A .

This is a repeated measure, nested-factorial design, with three factors (measurement system, left/right index finger, experimental step), instrument units nested within measurement system (two units for each system), and repeated measures over participants. 21 Table 1B ) and no distinction was made between the Welch Allyn reference units, which were used interchangeably. Each measurement period was 2 minutes in length and respiratory rate (RR) was collected. Participant demographic data, including age, height, weight, gender, ethnicity, hand dominance was collected. Patient participants were verbally screened for the following comorbidities as exclusion criteria: coarctation of the aorta, severe peripheral vascular disease, severe anemia, severe sickle cell disease and methemoglobinemia. However, none of the patient participants screened reported having these conditions. Table 1 . The order and device combination used to take simultaneous heart rate and SpO 2 measurements in: An out-patient participants (n=250) and B in-patient/ER participants (n=70)

Step

J o u r n a l P r e -p r o o f 9 A: Out-patient Setting (n=250) Erroneous readings with a recorded value of zero indicating a measurement was not obtained were reported and excluded from further analysis. The measurement error of an instrument is summarized by the root mean square deviation of prediction, RMSD, which is the square root of the mean square deviation (error) between the value reported by the instrument and the true value. The RMSD can be further decomposed into the bias and standard deviation (SD) of the instrument:

where bias refers to the systematic (average) deviation of the instrument from the true value, and SD summarizes the variation of the instrument in repeated measures 22 . Thus, bias measures accuracy and SD measures precision. The design of our clinical studies allows statistical analyses that tease out separately the bias and the SD of the two measurement systems and four units.

For the out-patient setting, accuracy and precision hypotheses were addressed by fitting linear mixed-effects models, to HR and SpO 2 outcomes. The model equation is Table 1A , but centered at 0; is subject-specific random effect, common to all measurements of participant i; and is within-participant error, assumed to have a mean 0. Furthermore, the within-participant errors were allowed to be autocorrelated, with an autoregressive of order 1 (AR1) structure, and to have variances that differ between groupsspecifically, test and reference systems or the four measurement units 23 .

The predicted biological value of outcome (HR or SpO 2 ) for generic subject i is + . Similarly, =±1/2 so that the bias between systems compares the average of the two test units versus the average of the two reference units, and so that measures the bias between the two test units.

Step are not central to our hypotheses, but they act as potential confounders, and may help in estimating the bias and precision of the instruments. These two factors were excluded if not significant at the 0.20 level in backward model selection 24 . The statistical comparisons and 95% confidence intervals used the Wald test for the mean (accuracy) coefficients, and the likelihood ratio test for the standard deviation (precision) parameters. The choice between AR1 and independence correlation structure for within-participant errors used the likelihood ratio test. A sensitivity analysis was performed, removing outlier observations defined as ≥ 3 times their system-specific standard deviation.

These large outliers are likely due to the incorrect use of the system (e.g., finger placement), and the sensitivity analysis results reflect the accuracy of the two systems under appropriate use conditions.

For the In-patient setting respiratory rates were summarized, with accuracy and precision evaluation simplified to two steps using a single test and reference unit based on a similar mixedeffects models as for the out-patient setting but without terms for units, T i and R i . Due to the small number of outliers for this setting a similar sensitivity analysis is not reported.

Statistical significance is considered at p<0.05 level. All analyses were conducted using the R statistical language 25 and used the nlme package 26 .

Test data was collected from 10 participants during 'breathe down' testing (see Methods), comparing in-phone sensors and Massimo Radical pulse oximetry device SpO 2 values. A Bland Altman Plot of the entire data set with second order polynomial line fit is shown in Figure 2 .

Below. In this study 8 plateaus were rejected for lack of stability within 2 SpO 2 Counts, rejecting 12 the corresponding 16 data points. All 10 subjects completed the study and there were no adverse events.

The root mean square deviation (RSMD) total average for data from all 10 volunteer participants tested was 2.02%. Heart rate error based on analysis of Fluke simulation testing showed that the heart rate error was < 2 bpm RMSD. 

Participants demographics confirmed a broad age range see Table 2 . (Table 4 ).

In the In-Patient/ER Setting for HR, no significant bias (accuracy) was found for the test relative to the reference (Table 3, Figure 3A ). For SpO 2 , test units had significantly lower readings than reference units, ( Table 3 , Figure 3B ).

In the Out-patient setting for HR, the test measurement system had significantly higher variation (worse precision) than the reference system (standard deviation (SD) = 5.99 bpm, 95% CI (5.61, 6.40) bpm vs. 3.80 bpm, 95% CI (3.56, 4.06) bpm, p<0.001, see Table 3 and Figure 3 ). The precision differed significantly between the two test units, SD = 6.92 bpm for test unit 1 and 4.95 bpm for test unit 2, p<0.001.

The precision did not differ significantly between the two reference units, p=0.27 (Table 4) .

Similar findings were observed in the In-patient/ER setting (See Table 4 ).

For SpO 2 , the test measurement system had significantly higher variation (worse precision) than the reference system, SD=1.25 % points, 95% CI (1.17, 1.33) points vs. 0.95, 95% CI (0.89, 1.01) points, p<0.001 (Table 3 and Figure 3 ). The precision did not differ between the two test units, 1.26 vs 1.25 points, p=0.90, however it differed significantly between the two reference units, SD = 1.01 and 0.86 points, p=0.001 (Table 4 ). Similar findings were observed in the In-patient/ER setting (see Table 3 , Table 4 ) and Figure 3A &B). Table 3 . Comparison of bias (accuracy), standard deviation (precision), and root mean square deviation of the Test (In phone) and Reference (Welch Allyn Spot Vital Signs) measurement systems, for heart rate and SpO 2 , in out-patient (n=250) and in-patient (n=70) studies. The

Reference system by definition has no bias and RMSD = SD. Bias and standard deviation correspond to and SD(ε ij ) in model equation (2) Test (In phone) and Reference (Welch Allyn Spot Vital Signs) measurement systems, for a) heart rate and b) SpO 2 , in the out-patient (n=250) and in-patient (n=70) studies. The error bars correspond to 95% confidence intervals. RMSD decomposes into SD and bias (See Methods, formula 1). The Reference system has no bias, by definition, and thus RMSD equals SD.

Results from certified laboratory testing indicate the smartphone sensor with App tested met ISO and FDA standards for pulse oximetry for reflective sensors 16, 17 compared to FDA/ISO approved reference. The testing facility calibrates and tests professional medical pulse oximeters before being cleared for sale by US FDA and other worldwide regulatory bodies. Full FDA/ISO approval standards also prescribe the testing we report in addition to at least 200 data points referenced to blood sample analysis.

The smartphone sensor with APP tested is embedded within approximately 300 million smart phones worldwide currently. 3 To evaluate whether this system could be utilized by a wide range of people to provide reliable, robust clinical measurements, we used a low risk, inexpensive, but rigorous protocol and statistical analysis to compare measurement accuracy and precision to hospital medical reference devices. Our methodology also allowed determination of accuracy and precision of measurements within each system. The model used a predicted biological value based on eight sets of measurements within the same individual to produce a 'true biological reading' free of bias associated with any one instrument.

In both out-patient and in-patient settings, the error of measurement (RMSD) was driven by measurement precision, with bias having a minor role. In the out-patient setting we found a small positive bias in the accuracy of HR and SpO 2 measurement by the smartphone sensor with App, which disappeared in the sensitivity analysis of HR, but persisted in SpO 2 measurement.

Precision of the smartphone sensor with APP was slightly less than reference units for HR and SpO 2 , a difference that persisted only for SpO 2 following sensitivity analysis. In the hospital/ER setting the accuracy for HR was the same for both 'test' and 'reference' systems. SpO 2 measurement however showed a small but consistent bias. However, our analysis revealed similar significant differences in accuracy and precision between the two Welch Allyn reference devices. This provides clarification that the consequence of small but significant test unit differences detected are of no clinical importance, and tolerable within high level medical settings. The reference instruments use FDA approved devices from two different manufacturers, Massimo (Irvine, CA) and Nelcor (Minneapolis, MN), that noninvasively measure the oxygen saturation by a red and infrared light source, photo detectors, and a probe to transmit light through a translucent, pulsating arterial bed in the digit; this may explain observed measurement differences.

Clinical investigations used an inexpensive plastic cell phone case to allow guidance of the digit over the smartphone sensor with APP. For adequate measures of SpO 2 , exposure of the sensor to maximal areas of capillary circulation in the finger pulp is essential and the digit needs to remain immobile for approximately 30 seconds. Our data indicate the precision (measurement variation) of smartphone pulse oximetry was worse when participants had to hold their finger on the sensor for two minutes (in-patient/ER setting), likely explained by difficulty associated with maintaining a digit in place for two minutes. In our experience PPG signal should be reliably obtained within 30-60 seconds of finger placement. The smartphone sensor with App tested in this study had no refinements for assessing PPG quality or exclude meaningless readings.

Improved App software, performing basic sensitivity analyses allowing evaluation of finger contact or outlier values necessitating repeated measurements, could ensure even greater ease of use and reading reliability. In the real world, people will use the smartphone embedded biosensor plus App as a spot checker, it is designed for the user to hold their finger in place for approximately 30 seconds and get a single reading.

Smartphone sensors with Apps enabling clinical pulse oximetry over multiple geographical locations on demand is of immense practical medical importance globally, particularly in LMICs. During respiratory viral pandemics smartphones sensors with Apps allowing clinical pulse oximetry may support medical practitioners in triage and initial clinical management. 14, 15 Moreover, smartphone sensors with Apps could easily be connected to regional hospitals and national data hubs, allowing large scale regional data analysis in near-real time to estimate resource requirements, and may empower patients by storing intermittent historical remote data to present to hospital triage or medical practioners, and could support management of chronic cardiopulmonary diseases and postoperative clinical follow-up care, including lung transplantation.

Our data shows the smartphone sensor with App tested met laboratory FDA/ISO standards and could be used to obtain highly accurate and repeatable measurements across a varied population. Full FDA/ISO approval would require additional laboratory testing to incorporate at least 200 data points referenced to blood sample analysis, which could be completed in a few weeks. Given the immediate practical medical importance of remote intermittent clinical pulse oximetry, Industry should be encouraged and supported to pursue full J o u r n a l P r e -p r o o f 20 FDA/ISO approval of this smartphone sensor with App. This approval should be prioritized and fast tracked by the FDA/ ISO organizations.

The data files are held by UCSD in a data repository. For access, please email the AVRC Regulatory Group: avrcregulatory@ucsd.edu

Specialists in Global Health (SiGH) (https://sigh.global) initiated this collaboration and brought the team that produced this research together, having identified the potential contribution to global health this work could make. The work was funded by a NIH grant supplement R01MH110057-04S to SHBrowne, a grant from Maxim Integrated Inc to SPan and a Technology Award from Specialists in Global Health (SIGH). 

Smartphone-based respiratory rate estimation using photoplethysmographic imaging and discrete wavelet transform

Universal Access to Essential Vital Signs Monitoring

Available at

Mobile Monitoring and Reasoning Methods to Prevent Cardiovascular Diseases

Continuous remote monitoring of COPD patients-justification and explanation of the requirements and a survey of the available technologies

A Real-Time Health Monitoring System for Remote Cardiac Patients Using Smartphone and Wearable Sensors

Prevalence of undiagnosed hypoxemia in adults and children in an underresourced district hospital in Zambia

Development of a clinical prediction rule to diagnose Pneumocystis jirovecii pneumonia in the World Health Organization's algorithm for seriously ill HIVinfected patients

Cost Effectiveness of Noninvasive Oxygen Saturation Measurement During Exercise for the Diagnosis of Pneumocystis carinae Pneumonia

Pulse oximetry for children with pneumonia treated as out-patients in rural Malawi

Hypoxaemia as a Mortality Risk Factor in Acute Lower Respiratory Infections in Children in Low and Middle-Income Countries: Systematic Review and Meta-Analysis

Evaluation of tuberculosisassociated chronic obstructive pulmonary disease at a tertiary care hospital: A case-control study

Chronic lung disease in adult recurrent tuberculosis survivors in Zimbabwe: a cohort study

Initial Clinical Management of Symptomatic Adult Patients during Influenza A (H1N1) Epidemics

Pulse-oximetry accurately predicts lung pathology and the immune response during influenza infection

ISO -International Organization for Standardization

Pulse Oximeters -Premarket Notification Submissions

Available at

Errors in 14 pulse oximeters during profound hypoxia

Gadgeteering for Health Care

Agreement Between Methods of Measurement with Multiple Observations Per Individual

Design and analysis of experiments

The elements of statistical learning, second edition: data mining, inference, and prediction

Mixed-effects models in S and S-PLUS

Simulation Study of Confounder-Selection Strategies

R: A language and environment for statistical computing. R Foundation for Statistical Computing

nlme: Linear and Nonlinear Mixed Effects Models

Ref -reference; bpm -beat per minute; Test Units -Smartphone sensor with App

COPD (chronic obstructive pulmonary disease)

FDA (Food and Drug Administration)

PPG (photoplethysmography)

Severe Acute Respiratory Syndrome Coronavirus

SpO 2 (Oxygen Saturation)

Competing Interests C-CH and CAE were employees of Maxim Integrated during the course of this study. The remaining authors declare no competing interests. J o u r n a l P r e -p r o o f J o u r n a l P r e -p r o o f Table 3 . Comparison of bias (accuracy), standard deviation (precision), and root mean square deviation of the Test (In phone) and Reference (Welch Allyn Spot Vital Signs) measurement systems, for heart rate and SpO 2 , in out-patient (n=250) and in-patient (n=70) studies. The

Reference system by definition has no bias and RMSD = SD. Bias and standard deviation correspond to and SD(ε ij ) in model equation (2)