key: cord-0735686-gm95ca92 authors: Tewolde, S.; Costelloe, C. E.; Powell, J.; Papoutsi, C.; Reidy, C.; Gudgin, B.; Shenton, C.; Greaves, F. title: An observational study of uptake and adoption of the NHS App in England date: 2022-03-20 journal: nan DOI: 10.1101/2022.03.16.22272200 sha: be641b2afbe06fe66ce8c9eef0108bfd0cb9ddfa doc_id: 735686 cord_uid: gm95ca92 Objectives: This study aimed to evaluate patterns of uptake and adoption of the NHS App. Data metrics from the NHS App were used to assess acceptability by looking at total app downloads, registrations, appointment bookings, GP health records viewed, and prescriptions ordered. The impact of the UK COVID-19 lockdown and introduction of the COVID Pass were also explored to assess App usage and uptake. Methods: Descriptive statistics and an interrupted time series analysis were used to look at monthly NHS App metrics at a GP practice level from January 2019-May 2021 in the population of England. Interrupted time series models were used to identify changes in level and trend among App usage and the different functionalities before and after the first COVID-19 lockdown. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines were used for reporting and analysis. Results: Between January 2019 and May 2021, there were a total of 8,524,882 NHS App downloads and 4,449,869 registrations. There was a 4-fold increase in app downloads from April 2021 (650,558 downloads) to May 2021 (2,668,535 downloads) when the COVID Pass feature was introduced. Areas with the highest number of App registrations proportional to the GP patient population occurred in Hampshire, Southampton and Isle of Wight CCG, and the lowest in Blackburn with Darwen CCG. After the announcement of the first lockdown (March 2020), a positive and significant trend in the number of login sessions was observed at 602,124 (p=0.004)** logins a month. National NHS App appointment bookings ranged from 298 to 42,664 bookings per month during the study period. The number of GP health records viewed increased by an average of 371,656 (p=0.001)** views per month and the number of prescriptions ordered increased by an average of 19934 (p<0.001)*** prescriptions per month following the first lockdown. Conclusion: This analysis has shown that uptake and adoption of the NHS App was positive post lockdown, and increased significantly due to the COVID Pass feature being introduced, but further research is needed to measure the extent to which it improves patient experience and influences health service access and care outcomes. research is needed to measure the extent to which it improves patient experience and influences health service access and care outcomes. The increasing adoption of electronic medical records (EMRs) by hospitals and GPs have given patients digital access to their personal health information through the utilization of patient portals. Patient portals provide the potential for improved patient-provider communication via electronic messaging, direct appointment booking, prescription ordering, and provide users a platform to be more informed about their health. 1 Some preliminary evidence has shown that patient portals may improve health outcomes and reduce health inequities, but the direct benefits are still unclear. 9, 10, 11, 12 There are also issues around trust, security, communication and interoperability with these new technologies. 19 Work by the OpenNotes team in the US suggests that offering patients access to doctors' notes is acceptable, improves communication and patients' confidence in managing their care, but also finds many clinicians are still resistant to the idea. 20, 21, 22 Recent reviews on patient portals show their widespread use in other countries and that patients' interest and ability to use them is influenced by age, ethnicity, education, health literacy, and health status. 23 One example of an established patient portal in the United States, is the Kaiser Permanente Northern California (KPNC)'s inpatient and ambulatory care electronic health record system. KPNC's 3.4 million patient portal users use the platform to exchange messages with their provider, create appointments, refill prescriptions, and view their medical records. 13 Other national patient portals have been established in Denmark, Estonia and Australia, where all citizens currently have access to their personal health data, but there is still uncertainty surrounding the impact on these portals and care outcomes and patient experience. 24 The NHS App In January 2019, the National Health Service (NHS) in England introduced a new app for patients, called "the NHS App". The enabled functions of the App for general use at launch were: 1. Check symptoms using NHS 111 online and the health A-Z on the NHS website 2. View their GP medical record 3. Book and manage appointments at their GP practice 4. Order repeat prescriptions 5. Register as an organ donor 6. Set data sharing preferences for the national data: choose if data is shared for planning and research In light of the novel coronavirus (COVID-19) pandemic, a new feature called the NHS "COVID Pass" was announced on April 28, 2021 and was then launched on May 17, 2021 under the name "NHS COVID Pass". The COVID Pass allowed users to prove their vaccination status in England, and received great interest in the UK media during the launch. 25 Users were also able to use the "Check your COVID-19 vaccine record" feature on the App which allowed patients to see which vaccines they had received historically. With over 3,000,000 mobile health applications currently on the market, a gap remains in identifying validated digital health solutions and their impact on public health. NHS England's goals for the App are to 1) improve access to primary care services, 2) improve patient experience, 3) save time in GP practices and 4) promote self-care. Our research aims to evaluate the effectiveness of the NHS App in meeting its goals by looking at early patterns of uptake and adoption. We will use observational data from the NHS App to assess use and acceptability by looking at total app downloads and registrations, total number of appointments bookings, total number of GP health records viewed, and the number of prescriptions ordered. We will also look at the change in frequency amongst these functionalities due to the impact of COVID-19 and the introduction of the COVID Pass. Data from the NHS App were analyzed using data metrics from the NHS App Dashboard. 18 The NHS App Dashboard was jointly developed by NHS England and Improvement team and NHS Digital under England's National Health Care system. 15 In this study, aggregate counts of NHS App activity were analyzed using data sourced from the NHS App Dashboard. 18 The NHS App Dashboard was jointly developed by NHS England and Improvement team and NHS Digital. 15 Metrics currently available through the dashboard include aggregate counts of the number of registrations, downloads, appointment bookings and cancellations, GP health records viewed, prescription requests, visits to NHS 111 online, organ donation registrations and withdrawals, visits to the health A-Z page, and visits to the national data opt-out page. These metrics can be viewed on a weekly or monthly basis and can be broken down geographically, by NHS Regions, Sustainability and Transformation Partnerships (STP) (planning framework for NHS services), Clinical Commissioning Groups 3 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) Anonymous analytics logs from the NHS App iPhone and Android applications service are collected and stored on a secure cloud service by NHS Digital. The NHS App data team at NHS Digital then generates an aggregate GP practice level count of app activity on a weekly/monthly basis. 18 The aggregate data are simply counts of the number of times each app feature was used in a given geography. These aggregate data are then used to develop the NHS App dashboard which is deployed on the NHS England Applications platform. At no point is patient level data made available or used by analysts at NHSX or Imperial College London. The research team did not have access to any personal or personally identifiable information. All NHS App metrics and descriptions are included in the Appendix. The study period for this analysis was from January 2019 to May 2021. Descriptive statistics were used to summarize monthly NHS App metrics at a GP practice level for the entire country. An absolute monthly count was presented from January 2019 to May 2021 of total App downloads, registrations, login sessions, appointments booked, GP health records viewed, and prescriptions ordered. New App downloads and registrations were also presented during the same time frame. A heat map extracted from the NHS App Dashboard displays the total percentage of the GP patient population aged 13+ registered for the App across the whole country. Lowest and highest App registrations by CCG were also presented for the population of England. An ecological time series and an interrupted time series (ITS) analysis was used to analyze the impact of the first UK national lockdown due to COVID-19 that occurred on March 26, 2020. The time series analysis was used to evaluate changes in uptake and the longitudinal impact of the pandemic on different functionalities of the App. 7 An interruption was added to the time series on April 1, 2020, to estimate changes in uptake before and after announcement of the first lockdown. The ITS explored changes in level and trend for national App logins, appointment bookings, GP health records viewed, and prescriptions ordered between January 2019 and May 2021. For each App metric, a linear regression 4 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted March 20, 2022. ; https://doi.org/10.1101/2022.03.16.22272200 doi: medRxiv preprint model with a time series specification was used, where pre-lockdown was denoted by "0" and the post-lockdown period was denoted by "1". The ITS model was then used to assess if there was a change in App usage immediately after the first lockdown (the level) and if a change in trend occurred over the whole study period. Absolute and relative changes of the post intervention trend were also estimated, had the first lockdown not taken place. The absolute change in trend was calculated by taking the difference between the predicted pre-lockdown trend of the outcome and the post-lockdown trend at the end of the study period. The relative change was calculated as the absolute change as a relative proportion. To analyze App usage and uptake before the NHS COVID Pass was introduced, a separate ITS analysis was conducted excluding the month of May 2021. Autocorrelation was assessed by analyzing the residuals of the ITS models. 8 The autocorrelation function plots (ACF) and partial autocorrelation function (PACF) plots of the residuals are included in the Appendix. The time series model for App metric Y at time t had the form: Where for each app metric Y, CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (Figure 3) . The ITS showed that before implementation of the first lockdown, the number of NHS App login sessions was increasing over time at a rate of 52,093 logins a month, and decreased immediately after implementation of the first lockdown. However, only the change in trend over time was found to be positive and significant, indicating an average slope change of 602,124 (p=0.004) logins a month. 12 months after the first lockdown, the average number of NHS App logins was 5,663,224 more than would have been expected if lockdown did not occur. This represented a 440% increase. (Table 1) . Appointments booked using the NHS App ranged from 298 to 42,664 per month during the study period. There was a surge of appointment bookings in January 2020 (33,003 appointments booked), September 2020 (32,335 appointments booked), and the highest in May 2021 (42,644 appointments booked). There was a significant drop in appointment bookings after the first lockdown was announced in April 2020 (7,674 appointments booked). The ITS showed that before the first lockdown, there was significant evidence of an increase in the average number of appointment bookings 2,247 (p<0.001), followed by an immediate decrease in bookings after the lockdown was announced -21,315 (p<0.001). 12 months after the first lockdown, the average number of appointments booked using the NHS App was 21,601 fewer than would have been expected if lockdown did not occur. This represented an overall 38% decrease in appointment bookings. Time series plots and the ITS analysis for NHS App login sessions and appointment bookings are depicted in Table 1 and (Figure 4) . The ITS showed that pre-lockdown, there was significant evidence that the number of prescriptions ordered was increasing at an average rate of 5,255 (p=0.001) a month. ( Table 2) . Post-lockdown, there was a positive and significant increase in trend of 19,934 (p<0.001) prescription orders per month. ( Table 2 ). 12 months after the first lockdown, the average number of NHS App prescription orders was 286,630 more than would have been expected if lockdown did not occur. This represented a 225% increase. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. significantly increased after the first lockdown, suggesting that more users were utilizing the App to place prescription orders rather than going in person or via the phone. These findings could be related to patients and carers being less likely to seek healthcare during the pandemic, in support of the pressure experienced by NHS services at the time, not wanting to bother the health service, and seeking alternative means to manage their health, either through self-management, or remote access (e.g. digital prescription ordering), as well as fear of getting infected. 27 App registrations across England varied by CCG, and no regional patterns were distinctly observed. Regions with the lowest proportion of App registrations seemed to be in some of the more deprived parts of England, which would merit further exploration at the level of individual data. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The public health impact of digital health interventions is dependent upon real-world uptake, and engagement. 17 Our analysis suggests that there is high uptake of the NHS App, but this may not always correspond with high levels of engagement. A recent study looking at uptake and engagement of health and well-being apps found that one of most important factors for engagement were apps coupled with health practitioner support. 17 Apps with a human support component were found to be more effective in increasing the effectiveness and engagement of digital health interventions. 17 At the time of this analysis, the NHS App does not directly incorporate any health practitioner support. The results of this observational study indicate that more research is needed to identify whether the NHS App is an effective digital health tool and the extent to which it has met the goals set out by NHS England. The existing literature on technology adoption and diffusion of innovation tells us that this process is difficult and complex. Adoption is not just based on the technology, but a complex mixture of how the public and staff interact with it, what they see as the benefits, organizational culture and wider influences on the system including the policy and regulatory context. 25 Our analysis showed that there has been strong adoption of the App, and that COVID has significantly expedited uptake, however, further research is needed to evaluate continuous usage of the App and if it yields any benefits. Our ecological analysis was only able to analyze the NHS App at the GP practice level, which did not allow us to make any inferences on individual app usage behavior. Having data only at the GP practice level data did not allow us to explore whether different subgroups were impacted by the App differently. Previous studies have shown that ethnic minorities, patients who are younger, healthier, and less educated were less likely to adopt to patient portals. 14 Person-level data of NHS App usage is needed to see if these same results are observed. Having person-level data would also further our evaluation by allowing us to assess whether there are any demographic and socio-demographic variations within App usage, and if the intervention has had any impact on clinical outcomes. The interrupted time series analysis also does not account for other confounding factors that may have contributed to the results seen in this study. This could include changes in seasonality, other existing co-interventions during the study period, and other events that may have occurred during the intervention. We tried to minimize known confounders by analyzing the data with and without the introduction of the COVID Pass, which has been included in the appendix. We intend to look at long term App usage once more data is available post-COVID. 16 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted March 20, 2022. ; https://doi.org/10.1101/2022.03.16.22272200 doi: medRxiv preprint As part of this NIHR-funded study, we are also conducting a qualitative assessment to understand the multiple interacting influences on uptake and use of the app from the perspective of patients, healthcare staff and commissioning and policy stakeholders. This is the first ecological study that has analyzed a nationwide intervention rolled out by NHS England. This analysis has shown that uptake and adoption of the NHS App is high, and has been driven by COVID related events, but further research is needed to measure the extent to which it improves patient experience and influences health service access and care outcomes. These results may inform other initiatives to harness patient-facing digital tools, and suggest improvements on future digital health interventions. Digital Health More than 10 Million People Now Using the NHS Digital health: a path to validation Challenges for the evaluation of digital health solutions-A call for innovative evidence generation approaches. npj Digit The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for Reporting Observational Studies | The EQUATOR Network Health and Social Care Act 2012 Health and Social Care Act 2012 Interrupted time series analysis in drug utilization research is increasing: systematic review and recommendations Using Patient Portals to Improve Patient Outcomes: Systematic Review Electronic patient portals: evidence on health outcomes, satisfaction, efficiency, and attitudes: a systematic review Electronic patient portals: evidence on health outcomes, satisfaction, efficiency, and attitudes: a systematic review Centre for Reviews and Dissemination (UK) Patient Portals Facilitating Engagement With Inpatient Electronic Medical Records: A Systematic Review And Video Technologies | Health Affairs Patient Portals and Patient Engagement: A State of the Science Review Mobile apps that have been used only once 2019 | Statista Influences on the Uptake of and Engagement With Health and Well-Being Smartphone Apps: Systematic Review Personal Health Records: A Systematic Literature Review US experience with doctors and patients sharing clinical notes VA OpenNotes: exploring the experiences of early patient adopters with access to clinical notes Inviting patients and care partners to read doctors' notes: OpenNotes and shared access to electronic medical records Patient Portals and Patient Engagement: A State of the Science Review Nationwide citizen access to their health data: analysing and comparing experiences in Denmark, Estonia and Australia Factors that influence the implementation of e-health: a systematic review of systematic reviews (an update) Using the NHS App as a covid-19 vaccine passport Do I, don't I?' A qualitative study addressing parental perceptions about seeking healthcare during the COVID-19 pandemic