key: cord-0014499-1c09v034 authors: Escriche-Escuder, Adrian; De-Torres, Irene; Roldán-Jiménez, Cristina; Martín-Martín, Jaime; Muro-Culebras, Antonio; González-Sánchez, Manuel; Ruiz-Muñoz, Maria; Mayoral-Cleries, Fermín; Biró, Attila; Tang, Wen; Nikolova, Borjanka; Salvatore, Alfredo; Cuesta-Vargas, Antonio I title: Assessment of the Quality of Mobile Applications (Apps) for Management of Low Back Pain Using the Mobile App Rating Scale (MARS) date: 2020-12-09 journal: Int J Environ Res Public Health DOI: 10.3390/ijerph17249209 sha: bbd5718d74d53362c79d09d707775d3b5286b3f8 doc_id: 14499 cord_uid: 1c09v034 Digital health interventions may improve different behaviours. However, the rapid proliferation of technological solutions often does not allow for a correct assessment of the quality of the tools. This study aims to review and assess the quality of the available mobile applications (apps) related to interventions for low back pain. Two reviewers search the official stores of Android (Play Store) and iOS (App Store) for localisation in Spain and the United Kingdom, in September 2019, searching for apps related to interventions for low back pain. Seventeen apps finally are included. The quality of the apps is measured using the Mobile App Rating Scale (MARS). The scores of each section and the final score of the apps are retrieved and the mean and standard deviation obtained. The average quality ranges between 2.83 and 4.57 (mean 3.82) on a scale from 1 (inadequate) to 5 (excellent). The best scores are found in functionality (4.7), followed by aesthetic content (mean 4.1). Information (2.93) and engagement (3.58) are the worst rated items. Apps generally have good overall quality, especially in terms of functionality and aesthetics. Engagement and information should be improved in most of the apps. Moreover, scientific evidence is necessary to support the use of applied health tools. Two reviewers (physical therapists with experience in lower back pain (LBP) management and mobile health (mHealth), previously trained in the use of The Mobile App Rating Scale (MARS)) searched applications (apps) that included an intervention for LBP in the official stores of the two main operating systems, Android (Play Store) and iOS (App Store). The search was carried out in September 2019. To maximise the chances of recovering all the target results, a general term such as "low back pain" was used for the search. Since each platform offered different content depending on the region, the search process was repeated on both platforms for localisation in Spain and the United Kingdom. Concerning this study, those applications (apps) related to lower back pain (LBP) that included feedback or intervention were selected. Apps that included other pain conditions with specific sections for LBP also were selected. Apps based solely on general pain or that were in languages other than English or Spanish were excluded. Likewise, apps with significant technical problems and paid apps also were excluded. Apps that were free to download and use but included exclusive paid content were not discarded, since they allowed the app to be used and evaluated. However, any free apps that subsequently required a subscription or that only allowed an initial assessment before requiring payment were excluded. The same independent reviewers screened the title and the download page of the found applications (apps). Potentially eligible apps were imported into a database (in case of doubt, they were imported), and duplicates from apps found in both regions were identified and unified. Remaining apps were downloaded, and any apps that did not meet the selection criteria were deleted. A third reviewer was consulted in the event of any doubts concerning the eligibility of an app. Two reviewers (physical therapists with experience in lower back pain (LBP) management and mobile health (mHealth), previously trained in the use of The Mobile App Rating Scale (MARS)) independently downloaded, used, and evaluated the remaining applications (apps), using an extraction form for data extraction. The form included data about the app developer, platform, version, year of publication of the last version, cost (paid or free app, and additional paid content), number of downloads, user rating, availability of privacy statement and privacy technical aspects, medical product information, and the items included in the MARS. The MARS [29] was applied to assess the quality of the included apps. This scale was based on 23 items grouped in the following five sections: (A) Engagement (five items): Fun, interesting, customisable, interactive (e.g., sends alerts, messages, reminders, feedback, enables sharing), well-targeted to the audience. (B) Functionality (four items): App operation, easy to learn, navigation, flow logic, and gestural design of the app. (C) Aesthetics (three items): Graphic design, overall visual appeal, colour scheme, and stylistic consistency. (D) Information quality (seven items): Contained high-quality information (e.g., text, feedback, measurements, and references) from a credible source. (E) Subjective quality (four items): Personal interest in the app Each item was scored from 1 (inadequate) to 5 (excellent), and a final score of app quality was obtained with the mean score of sections A, B, C, and D. The subjective app quality score was obtained independently with the average score of section E. Moreover, there were six app-specific items grouped in section F that assessed the perceived impact of the app on the user's knowledge, attitudes, intentions of change, as well as the likelihood of an effective change toward the health behaviour in question. Due to the exclusion of the subjective quality section from the overall mean app quality score, MARS's ability to assess quality objectively increased. Moreover, the high correlation between the MARS total score and the user's star rating found in previous studies suggested that it was capturing the perceived overall quality adequately [29] . Each reviewer independently assessed each of the Mobile App Rating Scale (MARS) items in each of the applications (apps). Regarding each item of each app, the mean value between the values of both reviewers was calculated. This mean value was used for the analysis of the mean of all the apps for each item and section of the MARS. Thus, average scores and standard deviations were retrieved from each section, as well as the total score and standard deviation of each app, allowing for a descriptive analysis. A third reviewer was consulted in the event of discrepancies in the descriptive items (items without a numerical score) between the two main reviewers. Following the example of a previous study [33] , apps were classified into tertiles to facilitate the interpretation of the readers. A total of 500 applications (apps) were retrieved from the Play Stores of the United Kingdom and Spain, while 53 apps were found in the App Stores of both countries. After deleting duplicates and screening the title and the download page of the remaining apps, 34 Android and 15 iOS apps were selected as potentially eligible from the Play Store and App Store, respectively. After downloading and checking the fulfilment of the selection criteria, 17 apps finally were included in the descriptive analysis. Three of the included apps were obtained from the App Store, while the remaining 14 apps were retrieved from the Play Store. Tables 1 and 2 , respectively, show the details and the main characteristics of the included apps. Ten apps included a privacy statement detailing which information was collected and for what purpose ( Table 2 ). Only six apps introduced login or password options to improve user's data privacy (Table 2 ). One app (Healo) declared to be General Data Protection Regulation compliant and secure in accordance with the European Union regulation and to be registered with the Swedish Medical Products Agency (Table 2) . Table 3 shows a summary description of how the apps work. The mean score of the users of the ten apps that reported these data was 4.11 (Standard deviation (SD) 1.07) on a scale of 1-5 stars. According to the Mobile App Rating Scale (MARS) scoring, mean app quality was 3.81 (SD 0.43), ranging from 2.83 (the worst rated) to 4.57 (the most highly rated). Table 4 shows the score of each app according to the MARS. Table 3 . Summary description of how the applications (apps) work. Back pain relief exercises The user chooses their pain area and receives a list of suggested exercises. There is a chat to contact health care advisers. It contains premium exercises unlocked by monthly payment. The app allows the user to set reminders. The application (app) offers examples of exercises and yoga poses to be performed, as well as tips for back care. Free access to sample exercises. The full content is available under subscription for three months. It allows the user to set reminders. The app offers examples of guided exercises depending on the objective selected by the user. It allows the user to set reminders. 6 Minute Back Pain Relief The app offers nine guided exercises related to yoga poses. It allows the user to set reminders. The app offers information related to back pain and guided exercises related to yoga poses. It allows the user to set reminders. The app offers guided exercises. It allows the user to configure the difficulty of the exercises and set reminders. The app includes extensive information related to anatomy, biomechanics, exercise, postural hygiene, lifestyles, diagnostic tests, and pain management strategies. It provides evaluation tests and questionnaires to be completed by the user that allows follow-up, as well as guided exercises. User can select favourite exercises. The app includes a section with utilities for the healthcare professional. The app offers guided exercises. It allows the user to configure the difficulty of the exercises and set reminders. The app contains some exercise programmes to be carried out in a guided way, as well as a battery of explained exercises. The user can self-evaluate. It allows the user to set reminders and configure the training frequency. Requires payment for full access to programmes. Back Doctor (FREE) Health. Stretch. Workout The app includes exercise programmes for neck and back pain, as well as chronic pain and other conditions. It provides information about lifestyles. Right Motion-Alivia tu dolor solo con ejercicios The app offers guided exercises. It allows the user to set reminders. The app includes exercise programmes with exercises described in text and video. Exercise programmes are personalised depending on the results of the diagnosis received by a doctor or through a self-assessment in the app. The app allows guided self-diagnostics that include the location (body chart) and the characteristics of the pain, its onset, or its duration, among others. It asks the user questions trying to find red flags. The app offers you a potential diagnosis and an associated exercise programme in which reminders can be set. It includes a chat with health care advisers. The design and graphs of the app are very attractive. The app includes explanatory videos of exercises and recommended postures in various situations of daily life. It allows the user to evaluate their pain to keep track of it. The app includes an exercise programme divided into progressive phases. The exercises are explained in writing, as well as in audio and video. It requires payment for access to full content. The app contains some exercise programmes to be carried out in a guided way, as well as assessments related to the pain and symptoms. It allows the user to schedule programme sessions and set reminders. It requires payment for access to full content. The app offers a narrative in the form of a written and listened conversation through which it discovers information related to pain. The intervention is focused on the neuroscience of pain and patient education. The design of the app is very attractive. The app contains some exercise programmes to be carried out in a guided way, as well as a battery of explained exercises. It allows user to set reminders. It requires payment for access to full content. This study aimed to review which mobile applications (apps) are currently available in the app market and to carry out an exhaustive assessment of their quality using the Mobile App Rating Scale (MARS). Described above in the Methods, the Mobile App Rating Scale (MARS) divides the application (app) scores into three dimensions. The assessment of sections A-D provides the mean quality score of the app. Subjective quality is scored according to the evaluation of section E. Finally, section F assesses the perceived impact of the app. This evaluation structure prioritises aspects such as engagement, functionality, aesthetics, and information over the subjective opinion of the assessor about the potential effectiveness of the app (E, subjective quality; F, perceived impact of the application). While digital health interventions require appropriate functional and design aspects, a high potential for impact and effectiveness are essential for continued use of the health tool. General assessment of the quality of the apps, therefore, should take into account the quality items assessed in sections A-D of the MARS, along with the score for subjective quality and app impact sections, as well as user rating. Moreover, not all aspects assessed on the scale have the same relevance for digital health interventions. Detailed analysis of the results in each section is essential in choosing an appropriate app. It is difficult to deduce the characteristics that an ideal app should have. Obtaining high levels of engagement through appropriate strategies that motivate behaviour change is essential, on the one hand. However, engagement is significantly influenced by the functionality and ease of use of the app and aspects such as aesthetics that make it more attractive. It is essential, on the other hand, that the information provided by the app is of quality to determine if its use will be safe. The efficacy of the intervention using each specific mobile health (mHealth) app should be studied, but this rarely occurs. Legally, the handling of the patient's private information will be a relevant aspect to know to comply with current regulations. During this study, the average quality of the 17 included apps ranged between 2.83 (Right Motion-Alivia tu dolor solo con ejercicios) and 4.57 (Healthy Spine & Straight Posture-Back exercises/Columna vertebral sana & Postura recta) (mean 3.82) according to the MARS, scored on a scale from 1 (inadequate) to 5 (excellent). Considering sections A-D, the best scores were found in functionality (mean 4.7), followed by aesthetic content (mean 4.1). To contrast, information (mean 2.93) and engagement (mean 3.58) were the worst rated items. Regarding subjective quality (mean 2.6) and app-specific assessment (mean 2.82), where impact on the user is most relevant, the mean scores were low. However, in these sections, the variability between apps was significant, with scores ranging between 1.25 (6 Minute Back Pain Relief)-4.25 (Healo) in subjective quality assessment and 1.83 (Yoga Poses for Lower Back Pain Relief; Bella's Lower Back Pain Exercises)-4.33 (Lower back yoga-floor class) in the specific items of the app, respectively. Overall, the mean score was good in most apps, with only one app (Right Motion-Alivia tu dolor solo con ejercicios) with a value below 3/5. However, the moderately low mean scores found in fundamental aspects such as engagement (3.58) and subjective assessment (2.6) should be analysed, due to the risk of developing technically well-designed applications that are unable to produce engagement and adequate adherence in users. Substantial differences were found between the quality score obtained using the MARS and the score awarded by users using stars. Concerning the case of user rating, the score for the apps was higher (mean 4.01) than in the assessment using the MARS (mean 3.82), except in three of them. The MARS score is an objective tool based on technical criteria considered necessary for an appropriate application of the intervention. Conversely, user assessment is usually subjective and, although it does not refer to specific items, it is mostly influenced by the impact upon and the subjective appreciation of the user. This fact makes comprehensive analysis of both assessment methods essential, otherwise technically well-designed apps could be overrated while those with aspects that were better subjectively valued by users could be dismissed. One of the foundations of tailored interventions is the premise that changes in sociocognitive determinants (attitudes, beliefs of efficacy) favour behavioural change [45] . There are different hypotheses about the mechanisms that underlie this process, but most of them agree that it is necessary to transform intention into action [46] . Education or provision of information seems to be the main strategy to influence these determinants [45] . Consequently, an intervention that provides details about the benefits of a specific healthy behaviour could persuade a person to take actions oriented toward this type of behaviour, which in turn could lead to a major change in behaviour. Other strategies to favour this behavioural change could be the use of feedback, monitoring, reminders, or the creation of a social group. The Mobile App Rating Scale (MARS) has a list of theoretical background/strategies that may have been applied in applications to promote behaviour change. These strategies are based on evaluation, feedback, information and monitoring, the choice of goals, and the proposal of advice and training methods, among other behavior change techniques. Additionally, it has a list of possible technical aspects that enhance these strategies. The evaluator marks the strategies used by each app. During this study, information and education, followed by the advice, strategies, and skills training were the most used strategies to motivate behaviour change. The most widely used technical aspect to favour these strategies was sending reminders. Health interventions supported by mobile health (mHealth) applications (apps) have experienced a significant increase. However, this rapid proliferation often hinders appropriate assessment of the effectiveness and validity of the applied tools [30] . Consequently, although studying the feasibility, safety, and effectiveness of implementing a health intervention is essential, most of them are applied despite a lack of such evidence [47] . The Mobile App Rating Scale (MARS) assesses whether there is a scientific basis (if the app has been trialled/tested and verified by evidence published in scientific literature) supporting the use of apps through an item included in the information section [29] . No scientific evidence supporting use of any of the apps included in this study was found. These results are in-line with previous reviews carried out on mHealth apps focused on pain, in which a significant lack of scientific basis was found to support the use of the available apps [33, 48] . Previously suggested by some authors, these results may be due to the commercial and non-scientific origin of the included apps [33] . It is possible that promoting the development of apps by academic and scientific institutions (and the collaboration with commercial partners) could help improve this aspect [33] . However, it seems that the long times required for research, with long delays with respect to the development time of new applications, could precisely be a possible cause for the lack of app evaluations [49] . Greater support of scientific and academic institutions with more resources to shorten the development and testing of this type of mHealth solution, therefore, could favour the proliferation of apps with evidence-based use. Health applications (apps) handle users' private information on health status, habits, or preferences. While any personal information must be handled with caution, health information is especially sensitive [50] [51] [52] [53] . The law protects patients' rights to the confidentiality of their health data, but such laws are usually drafted for application in the health system. Applying and monitoring compliance with these rights in mobile health (mHealth) apps is, therefore, a challenge [51] . The Mobile App Rating Scale (MARS) includes items about whether access controls have been introduced in the app using login or password options to improve privacy. However, these items are included in a section of technical aspects of the apps; this is merely for descriptive purposes and does not influence the quality score. During this study, six apps (Back pain relief exercises; Regimen-back pain relief; Healo; Bella's Lower Back Pain Exercises; Healure: Physiotherapy Exercise Plans; Curable: Back Pain, Migraine & Chronic Pain Relief) introduced login and password options. This may be essential as a first step to maintaining the confidentiality and security of user data. However, this information does not cover what happens with these data after collection. Storage and possible use of personal data by the app developers or third-party use are other fundamental aspects to analyse in the preservation of privacy and confidentiality. However, in the case of apps, the user does not usually have access to this information. A previous study found that most apps do not follow well-known practices and guidelines, threatening user privacy [50] . This circumstance could be improved if the developers informed users with privacy policy statements about possible uses of the user's information, but this statement is often absent [54] . Concerning our study, ten apps ( Safety in the use of applications (apps) that can be used for health purposes is an essential aspect for practitioners. Clear indications about those activities and exercises that can be performed autonomously at home, or those that should be performed supervised, must be indicated for an adequate prescription and use. Information about possible limits related to the health status of users is also essential. Some of them had an initial profile that allows the intervention to be personalised. However, the adequacy of this assessment is also unclear. Some of the apps included disclaimers, stating that the services offered are for informational purposes only and not for professional medical advice. Thus, five apps (6 Minute Back Pain Relief; Back Doctor (FREE) Health. Stretch. Workout; Curable: Back Pain, Migraine & Chronic Pain Relief; Yoga Poses for Lower Back Pain Relief; Escuela de Espalda) declared that the information provided by the app is not a substitute for a medical service, recommending visiting the doctor before starting the programme. Only one app, (Healo), declares to be secure in accordance with European Union regulation and to be registered as a medical product (Swedish Medical Products Agency). Developers should consider this aspect in future apps. The Mobile App Rating Scale (MARS) evaluates the information provided by the developers and the credibility of the source (conflicts of interest, commercial interest, academic origin, etc.). Additionally, MARS describes the evidence base of the apps (if the app has been trialled/tested and verified by evidence published in scientific literature). None of the included apps was found to be supported by evidence published in the scientific literature. This fact may be related to the commercial origin of all the included apps. Regarding this, the long times required for research, with long delays compared to the development time of new applications, have been attributed as a possible cause for the lack of app evaluation [49] . "Healo" is the only application (app) that declares to be registered as a medical product. Obtaining an average quality score of 4.2 (1.23), it is one of the best-rated apps in this study. It had high scores in the engagement, functionality, and aesthetics sections. Additionally, the subjective quality score and the perceived impact of the app on the user are high. However, the information provided in the app description is insufficient. The advanced self-diagnosis tools it contains, the availability of a chat with health care advisers, the adaptation of the programmes to each user, as well as its attractive design, make it a potentially useful mobile health (mHealth) app. "Healthy Spine & Straight Posture-Back exercises/Columna vertebral sana & Postura recta" is the app with the highest objective score using the The Mobile App Rating Scale (MARS) scale [4.57 (0.63)]. It is a well-configured functional app that allows one to carry out exercise programmes in a guided way. The app allows the user to self-assess and track the progress of the programme and the results of the assessments. However, this app requires payment to have full access to the contents. "Curable: Back Pain, Migraine & Chronic Pain Relief" is the only one of the included apps that does not offer an exercise-based intervention. Focused on patient education and pain neuroscience, it uses an attractive design through a narrative in the form of conversation, with text and audio. Obtaining an average objective quality score of 4.38 (0.82), it is one of the best-rated apps. It had high scores in the engagement, functionality, and aesthetics sections. Additionally, this app obtained high scores in the quality of the information and in the subjective perception of the quality and its impact on the user. The main limitation of this review is the exclusion of paid applications (apps). The search was carried out in duplicate in app stores in Spain and the United Kingdom. Therefore, apps available only in stores of other countries could not be retrieved. The app market is constantly updating. Although this may be a limitation, this review faithfully shows the current state of the market at the time of evaluation. This study offers an analysis and description of applications (apps) available for managing lower back pain (LBP), to help practitioners and users to recommend quality mobile health (mHealth) apps adapted to the needs of each patient. The assessed apps generally had good overall quality, especially in terms of functionality and aesthetics. However, some apps must improve aspects such as engagement and information to increase their impact on users and ensure better security and privacy. "Healo" and "Healthy Spine & Straight Posture-Back exercises/Columna vertebral sana & Postura recta" are mHealth apps with high objective and subjective quality scores that include an exercise-based intervention. "Curable: Back Pain, Migraine & Chronic Pain Relief" is another well-rated app that includes a neuroscience pain-based intervention programme. Moreover, additional scientific evidence is necessary to support the use of each mHealth app. Therefore, despite the extensive volume of apps on the market, this review shows an absence of validity studies, studies of psychometric characteristics, and clinical trials that demonstrate the effectiveness of these apps in people with LBP. This indicates the need to develop studies to allow the use of validated, effective apps supported by evidence. The safety of use and privacy also should be improved in most of the apps. Effects of a Pain Catastrophizing Induction on Sensory Testing in Women with Chronic Low Back Pain: A Pilot Study Epidemiological features of chronic low-back pain Randomized controlled trial of a 12-week digital care program in improving low back pain Global, regional, and national incidence, prevalence, and years lived with disability for 328 diseases and injuries for 195 countries, 1990-2016: A systematic analysis for the Global Burden of Disease Study Care for low back pain: Can health systems deliver? Clinical guidelines for low back pain: A critical review of consensus and inconsistencies across three major guidelines National Clinical Guidelines for non-surgical treatment of patients with recent onset low back pain or lumbar radiculopathy Non-Specific Low Back Pain Clinical Guidelines Committee of the American College of Physicians Noninvasive Treatments for Acute, Subacute, and Chronic Low Back Pain: A Clinical Practice Guideline From the American College of Physicians American College of Physicians Nonpharmacologic therapies for acute and chronic low back pain: A review of the evidence for an American Pain Society/American College of Physicians clinical practice guideline Exercise therapy for chronic nonspecific low-back pain Best Evidence Rehabilitation for Chronic Pain Part 3: Low Back Pain The association between physical activity and low back pain: A systematic review and meta-analysis of observational studies People with low back pain want clear, consistent and personalised information on prognosis, treatment options and self-management strategies: A systematic review Effect of Cognitive-Behavioral Training on Pain Self-Efficacy, Self-Discovery, and Perception in Patients with Chronic Low-Back Pain: A Quasi-Experimental Study Understanding the link between feelings of mental defeat, self-efficacy and the experience of chronic pain Literacy-Adapted Cognitive Behavioral Therapy Versus Education for Chronic Pain at Low-Income Clinics: A Randomized Controlled Trial Aerobic vs. resistance exercise for chronic non-specific low back pain: A systematic review and meta-analysis Exercise for chronic musculoskeletal pain: A biopsychosocial approach Patient-centeredness in physiotherapy: What does it entail? A systematic review of qualitative studies Intensive interdisciplinary treatment for a patient with coexisting pain and obesity: A case study The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: Building an international consensus for the reporting of behavior change interventions Tailor made for Health Psychology: Issues in the design and effectiveness of Internet interventions Tailoring Health Messages: Customizing Communication With Computer Technology; Routledge: London The relative persuasiveness of gain-framed and loss-framed messages for encouraging disease prevention behaviors: A meta-analytic review Efficacy of tailored-print interventions to promote physical activity: A systematic review of randomised trials Taking online computer-tailoring forward: The potential of tailoring the message frame and delivery mode of online health behaviour change interventions Examining the effectiveness of web-based personally-tailored videos to increase physical activity: A randomised controlled trial protocol Mobile App Rating Scale: A New Tool for Assessing the Quality of Health Mobile Apps Issues and considerations for healthcare consumers using mobile applications. Stud Evaluating mobile phone applications for health behaviour change: A systematic review Review and Evaluation of Mindfulness-Based iPhone Apps Measuring the Quality of Mobile Apps for the Management of Pain: Systematic Search and Evaluation Using the Mobile App Rating Scale Quality Evaluation of English Mobile Applications for Gestational Diabetes A systematic review of the smartphone applications available for coronavirus disease 2019 (COVID19) and their assessment using the mobile app rating scale (MARS) Assessing the quality of mobile applications targeting postpartum depression in China Quality of Deaf and Hard-of-Hearing Mobile Apps: Evaluation Using the Mobile App Rating Scale (MARS) with Additional Criteria From a Content Expert Mobile Apps in Rheumatology: Review and Analysis Using the Mobile Application Rating Scale (MARS) A review and content analysis of engagement, functionality, aesthetics, information quality, and change techniques in the most popular commercial apps for weight management Skin Cancer Smartphone Applications for German-speaking Patients: Review and Content Analysis Using the Mobile App Rating Scale Patient focused oral hygiene apps: An assessment of quality (using MARS) and knowledge content Smartphone Applications to Support Sleep Self-Management: Review and Evaluation Consumer Mobile Apps for Potential Drug-Drug Interaction Check: Systematic Review and Content Analysis Using the Mobile App Rating Scale (MARS) Development and Validation of the User Version of the Mobile Application Rating Scale (uMARS) Can you elaborate on that? Addressing participants' need for cognition in computer-tailored health behavior interventions Health promotion by social cognitive means Does evidence support the use of mobile phone apps as a driver for promoting healthy lifestyles from a public health perspective? A systematic review of Randomized Control Trials Commercially Available Smartphone Apps to Support Postoperative Pain Self-Management: Scoping Review Rapid, responsive, relevant (R3) research: A call for a rapid learning health research enterprise Security and Privacy Analysis of Mobile Health Applications: The Alarming State of Practice Privacy Assessment in Mobile Health Apps: Scoping Review Privacy in mobile technology for personal healthcare Exploring the Far Side of Mobile Health: Information Security and Privacy of Mobile Health Apps on iOS and Android Availability and quality of mobile health app privacy policies