key: cord-0961258-8a8b7pst
authors: Luo, Lingzi; King, Allison A; Carroll, Yvonne; Baumann, Ana A; Brambilla, Donald; Carpenter, Christopher R; Colla, Joseph; Gibson, Robert W; Gollan, Siera; Hall, Greg; Klesges, Lisa; Kutlar, Abdullah; Lyon, Matthew; Melvin, Cathy L; Norell, Sarah; Mueller, Martina; Potter, Michael B; Richesson, Rachel; Richardson, Lynne D; Ryan, Gery; Siewny, Lauren; Treadwell, Marsha; Zun, Leslie; Armstrong-Brown, Janelle; Cox, Lisa; Tanabe, Paula
title: Electronic Health Record–Embedded Individualized Pain Plans for Emergency Department Treatment of Vaso-occlusive Episodes in Adults With Sickle Cell Disease: Protocol for a Preimplementation and Postimplementation Study
date: 2021-04-16
journal: JMIR Res Protoc
DOI: 10.2196/24818
sha: 19ce8c7367970e2995e56201691e4670736d8c3b
doc_id: 961258
cord_uid: 8a8b7pst

BACKGROUND: Individuals living with sickle cell disease often require aggressive treatment of pain associated with vaso-occlusive episodes in the emergency department. Frequently, pain relief is poor. The 2014 National Heart, Lung, and Blood Institute evidence-based guidelines recommended an individualized treatment and monitoring protocol to improve pain management of vaso-occlusive episodes. OBJECTIVE: This study will implement an electronic health record–embedded individualized pain plan with provider and patient access in the emergency departments of 8 US academic centers to improve pain treatment for adult patients with sickle cell disease. This study will assess the overall effects of electronic health record–embedded individualized pain plans on improving patient and provider outcomes associated with pain treatment in the emergency department setting and explore barriers and facilitators to the implementation process. METHODS: A preimplementation and postimplementation study is being conducted by all 8 sites that are members of the National Heart, Lung, and Blood Institute–funded Sickle Cell Disease Implementation Consortium. Adults with sickle cell disease aged 18 to 45 years who had a visit to a participating emergency department for vaso-occlusive episodes within 90 days prior to enrollment will be eligible for inclusion. Patients will be enrolled in the clinic or remotely. The target analytical sample size of this study is 160 patient participants (20 per site) who have had an emergency department visit for vaso-occlusive episode treatment at participating emergency departments during the study period. Each site is expected to enroll approximately 40 participants to reach the analytical sample size. The electronic health record–embedded individualized pain plans will be written by the patient’s sickle cell disease provider, and sites will work with the local informatics team to identify the best method to build the electronic health record–embedded individualized pain plan with patient and provider access. Each site will adopt required patient and provider implementation strategies and can choose to adopt optional strategies to improve the uptake and sustainability of the intervention. The study is informed by the Technology Acceptance Model 2 and the Reach, Effectiveness, Adoption, Implementation, and Maintenance framework. Provider and patient baseline survey, follow-up survey within 96 hours of an emergency department vaso-occlusive episode visit, and selected qualitative interviews within 2 weeks of an emergency department visit will be performed to assess the primary outcome, patient-perceived quality of emergency department pain treatment, and additional implementation and intervention outcomes. Electronic health record data will be used to analyze individualized pain plan adherence and additional secondary outcomes, such as hospital admission and readmission rates. RESULTS: The study is currently enrolling study participants. The active implementation period is 18 months. CONCLUSIONS: This study proposes a structured, framework-informed approach to implement electronic health record–embedded individualized pain plans with both patient and provider access in routine emergency department practice. The results of the study will inform the implementation of electronic health record–embedded individualized pain plans at a larger scale outside of Sickle Cell Disease Implementation Consortium centers. TRIAL REGISTRATION: ClinicalTrials.gov NCT04584528; https://clinicaltrials.gov/ct2/show/NCT04584528. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/24818

Model Consent Document 1 1 1

Other materials 2 *Please describe the changes deemed necessary for approval:

1. Major concern is the primary outcome which is the change in the composite score of patient perception of quality of pain management in the ED setting. The investigators are going to use as a baseline patient recall of their experience in the ED in the 12 months prior to enrollment and compare that to a survey done within 72 hours of a study visit during the study period. I have two concerns with this: one is that I am not sure they are measuring the same thing as for the baseline patients are being asked to recall their experience. For some patients this might be recalling one episode of ED care and for others it could be recall of many ED visits. The investigators are then going to compare this recall assessment with a survey done at the time of a visit. The investigators state that a change of 0.5 SD would be clinically significant but it is unclear how they came to that conclusion.

The study team discussed the first concern at length. We chose to phrase the question of recall of 12 months, vs. the last visit, because many patients will typically remember only the "worst" visit, which in some cases, could have been 10 years prior. We have no guarantee that patients will interpret as we describe, but a script of how to ask this question when enrolling patients will be developed. This will be critical. We do think it is valid to compare a baseline, overall quality of pain management variable to measure changes with future visits. Essentially, patients are being asked to report, their perception of pain care over the past year, with a most recent visit, within 72 hours, for patients that have an ED visit after enrollment.

The study team, including the statistician, discussed the 0.5 SD change at length. Given the distribution of the scale, we agreed that a 0.5 change in SD would represent a significant change in the perception of quality of care.

The team has discussed this at length and due to the limitations of conducting this difficult protocol in an ED setting, we believe that obtaining patient reported perception of the quality of ED pain care within 72 hours of an ED visit is a more than acceptable proxy measure for pain relief. We are happy to discuss further.

As stated in the protocol "Quality of ED pain care in this study is the primary outcome because quality of pain care is reflective of how well the pain was managed and IPP's are the gold standard for VOE management. The 

Please refer to the definition of 'a qualified ED visit' on protocol page 16, we will only include data on ED visits for VOE reasons.

5. It appears that some patients already have pain plans. One has to assume that at least some of those pain plans have been used in the past so when those patients reflect on their prior 12 months in the ED it might be influenced by the fact that they have an existing pain plan-how will the investigators deal with this issue when looking at outcomes? Many sites do not already have pain plans, so new pain plans will be developed for several sites. The dataset will include a variable, "existing IPP", in the dataset. We will conduct an analysis to measure the effect of pre-existing plans. We have asked each site to report the number of patients with: no individual pain plans, IPP's written in a clinic note, IPP's accessible somewhere else in the EHR that is easily accessible.

6. What about patients who have restrictive pain plans (one that limit opioid exposure due to concerns over the risks of opioids v benefits for the individual patients? That might significantly influence the subjects' satisfaction with the quality of their pain management. Will those patients be included in this study?

Instructions: This form is designed to guide protocol reviews and to collect comments, issues of concern, requests for additional information or clarification, and to record an initial approval recommendation. Please forward the completed form to y 1. Second paragraph prior to section 2.2, sentence spanning the two pages "By assessing use of the IPP, we will be able to compare patient and provider outcomes, statistically, …" It is not clear how you will do this. Are you referring to tables 2 and 3 here and allowing X in your LMM equation to be a specific implementation component (likely the optional, as the required are going to be confounded with each other)? Please elaborate. Deleted

Study hypothesis(es) X Experimental design: sample size, use of controls or standard treatment or therapy, single/multicenter design X Statistical approach, power and data analysis plan X

Exploratory because we won't have enough power for this.

NHLBI / SCD-SCDID OSMB Protocol Review Form, Reviewer 4, Nov 26.

All sites are encouraged to participate in this project, thus we do not have this option, although it would be helpful.

While desirable, we do not have the resources to collect data at other EDs.

4. Is the difference of SD clinically meaningful? If you are averaging three questions together, a change of an outcome of 0.39 is essentially a one-unit increase on one of the questions. It's good to be able to power something that small, but I would hope a more clinically meaningful difference is defined. Addressed in other response.

Is there any expectation that the impact of the satisfaction of the IPP would vary according to how many visits there were to the ED for the participant. It is possible. Not enough power for the analysis. We will take the first visit of the patient in the stimulation.

We will measure missing data for the follow up surveys that are distributed after an ED visit. Will you use all data (to get better baseline estimates) or just participants who have a follow-up ED visit? We will use all data at baseline for baseline sample descriptions, but we will only be able to assess implementation and intervention outcome based on participants who have an ED visit for VOE during the implementation period.

7. Qualitative Data, Do sites have analysis teams? What is their membership? How many people per team? How are they trained? All analysis is done at each site, will be members of the research team, and will be trained by the RTI team.

Plans for study operations: organization of clinical centers, governing committees, data coordinating center X Plans for training clinical center personnel: good clinical practice, research goals, protocol details and implementation, and study operations X Plans for data acquisition: data management, quality assurance, and ultimate transfer of data to NHLBI X COMMENTS:

1. How will you prioritize the 5 provider and 5 patient interviews as you move forward in real time, within 2 weeks of the visit? Who will make that decision? If it is based on IPP use, then are you targeting mostly those who use the IPP, who don't use the IPP or a mix of both? Will this be with the aid of the DCC? If it is related to the change in the primary outcome, will that cause any conflict for the site to see change in outcome data (I know it isn't randomized)

We will recruit based on IPP use and target a mix of both. Sites will make recruitment decisions on their own, but we will have regular biweekly meetings to debrief on the recruitment process.

2. Prior to tracking and reporting implementation activities and strategies. I'm a bit confused as to the difference between provider and participant surveys and the ancillary study described in Appendix A.Please clarify.

The patient and provider survey and interview is to assess implementation and intervention outcomes. The parallel study is to track and report implementation strategies and their specifications used by each site, which will include interviews of site PI or research staff.

4. Provider participants -If you are not getting written consent for the surveys, will you be able to link the pre-post surveys together (or the survey of the provider at the beginning of the study to the post-SCD participant ED contact survey)? Yes we will establish a participant ID for providers.

This is not directly related to the protocol but important information.

6. RE-AIMTable . I'm confused by this one. Aren't written IPPs within the EHR the definition of "adopted" for the IPP? What does "after study targets are met" mean? 

Our primary aim is to assess patient's satisfaction of ED pain treatment through their perceived quality of ED pain treatment. A recent systematic review published in 2018 identified 24 instruments used between 1997 and 2017 assessing patient reported outcomes (PRO) in populations with SCD (both adults and children) in the U.S. 1 Five of the 24 instruments were developed for adult SCD patients. 1 The Adult Sickle Cell Quality of Life Measure (ASCQ-Me) Quality of Care (QOC) measure was the only one that assesses treatment satisfaction. 1 It was rated as having both good content validity and internal reliability. 1 Unfortunately, none of the 24 PRO instruments provided information regarding threshold of minimally important change. 1 With its limitation in mind, the workgroup decides that ASCQ-Me QOC is the best available instrument to use for the study population and our primary aim.

The ASCQ-Me QOC measure was modeled after the Consumer Assessment of Healthcare Providers and Systems (CAHPS) surveys, a widely used tool by the Centers of Medicare and Medicaid Services (CMS), Department of Defense and Department of Veterans Affairs. [2] [3] [4] There is evidence of the reliability and validity of the instrument and the ASCQ-Me QOC measure has been used by other research studies. 3, 5 The ASCQ-ME QOC includes global evaluation of care for SCD received from regular provider and the ED, and QOC in three domains: provider communication, ED care, and access. 3 To avoid participant survey fatigue and ensure we collect information that is most helpful and relevant, the workgroup chose the three questions from the ASCQ-ME QOC to measure perceived quality of ED pain treatment. Question 1) is to address global evaluation of care for SCD in the ED.

Within the timeframe of the SCDIC grant, we do not have time to conduct a rigorous psychometric testing of the items prior to initiation of the project. However, we believe the ASCQ-Me QOC validation that was conducted is sufficient as it identified the construct of "quality of care" in ED, and the "care" we are providing is pain management. As no instrument has provided a threshold of minimally important change, after numerous discussions, the workgroup has reached the conclusion that a 0.5 SD change will be a significant change.

Over the past 18 months, the workgroup has explored possibilities of using other patient outcomes as our primary outcome, but eventually we decided that they were not as feasible or not as reliable compared to assessing patient satisfaction, which is also an important patient outcome. The table below suggests other possible measures of pain management and rationale as to why they were not selected as the primary outcome.

Reason of not choosing as the primary outcome Visual analogue scale (0-10) -prospectively obtained from patient during treatment for pain. (gold standard)

Sites are not equipped to obtain pain ratings during the ED visit. EDs typically have only 1-3 VOE visits/day. Given the other expectations of the SCDIC grant (ongoing registry data collection and participation in a 2 nd and possible 3 rd implementation science project), it is not possible to have research staff available 24/7 to obtain this data.

Requiring ED providers to have that rating will add significant implementation fidelity challenges and is not feasible. Visual analogue scale (0-10) -documented in the EHR There is tremendous variability as to "if and when" pain scores are documented. Reliant on the EHR would result in a very large amount of missing data for the primary outcome.

A proxy measure of the effectiveness of pain management. We originally proposed this as the primary outcome, however the sample size was over 100 subjects/site. When proposed to the sites, all 8 sites stated they would not be able to participate in the project as it was not feasible to enroll 100 participants at each site.

The workgroup values assessing implementation outcome. Table 4 of the protocol has listed many implementation outcomes that we will measure. However, the workgroup aims to demonstrate that the IPP has a positive impact on patient care. Having a PRO instead of implementation outcomes as our primary outcome is the best approach.

We acknowledge that there was a concern of recall bias at baseline, and comparing past 12 months satisfaction at baseline and last visit satisfaction at follow-up. To address these concerns, we propose a change in our inclusion criteria: we will recruit individuals with SCD who have a participating site ED VOE visit within the past 90 days. We will also change our baseline assessment to perceived quality of care of the last ED visit (happened within 90 days), instead of for the past 12 months. With this revision, we will be able to compare baseline last visit satisfaction with follow-up last visit satisfaction. This change is summarized in the table below.

We may need additional recruitment strategies. If recruiting via phone, the team needs to schedule an orientation, during which the team member will help patients install patient portal app on the phone, show them how to access the IPP, and let patients show the staff how to access (a teach-back method). Some patients who have a recent ED VOE visit will have a scheduled clinic visit within a few weeks that could serve as the orientation. The research teams will make efforts in increasing the show up rate of the orientation. Study team can also screen patients who have scheduled clinic visits and recruit those who have an ED VOE visit within 90 days at the clinic.

We propose to set the eligibility criteria of having an ED VOE visit within the last 90 days, instead of 30 days, based on work group discussions and our registry data. Using registry data from one of the study site, Washington University in St. Louis, as an example: the site maximized recruitment efforts and approached all eligible patients with a clinic visit from Sep 2017 to Jan 2019 (18 months), enrolled 271 patients from clinic visits (decline rate less than 3%). At the time of enrollment, 35 (12.9%) had an ED VOE visit within 30 days, and 105 (38.7%) had an ED VOE visit within 90 days. This is for a 18 months enrollment period, so for this proposed study which has a much shorter enrollment period, setting the eligibility criteria of having an ED VOE visit within the last 30 days will make it impossible for sites to reach enrollment target (e.g. Washington University will be able to recruit less than 15 patients over a 6 months enrollment period from the Clinic). After OSMB reviews, sites are expected to have additional recruitment strategies, however, recruiting from the clinic or from the registry will be the most effective recruitment strategies (recruiting from the ED, many individuals will not be eligible if they did not have a participating site clinic visit within the past 12 months and/or the hematologist is not able to build a pain plan for them). So we decided to set the eligibility criteria to having an ED VOE visit at participating EDs within the last 90 days, instead of 30 days.

Additional major changes to the protocol:

1. Changed follow-up time window from 72 hours to 96 hours to give research staff time to follow up with patients with an ED VOE visit on Friday.

2. Added a new criteria for the qualified ED visit: the first participating ED VOE visit of the month (p.17). The rationale of this criterion is to alleviate the burden of data collection and retrieval if the patient has more than one visit in a month. Instructions: This form is designed to guide protocol reviews and to collect comments, issues of concern, requests for additional information or clarification, and to record an initial approval recommendation. Please forward the completed form to y February 17, 2020 1. Can the investigators clarify where they are collecting data on whether what the participant received in an ED visit was consistent with the IPP (I may have missed it but it doesn't look like it is in the data extracted from the ED visits). I see the providers are asked about whether they followed the IPP but collecting the objective data that the plan was followed would be important. What if a provider says they followed it but in fact it was not followed. The charts are being extracted anyway so would be fairly easy to collect. This information will be extracted in the EHR retrieval data. 2. For data analysis plan. I believe you are trying to show a three-level mixed effects model. Your random intercept includes the overall mean (\mu), the random site effect (lpha_j) and the nested individual in site effect (

u_{ji}). However, I do not agree with the parameterization of your slope. \Beta X is the fixed slope. The random slope should be 	au_j X. I do not believe that the random effect for site should equal the random effect for the slope (hence the lpha_j on the slope should be 	au_j). I also do not believe you need the eta with the 	au_j X term. Then the individual error is psilon_{jti}. "because the site is a random effect, the interaction is also a random effect". I do not believe that the slope needs to be random because the intercept is random. It can be that way, but it doesn't statistically need to be that way.

Quantitative data analysis plans have been updated please see track changes. Experimental design: sample size, use of controls or standard treatment or therapy, single/multicenter design Statistical approach, power and data analysis plan 1. Regarding the "patient eligibility criteria #1, diagnosis. If no medical record is available are the sites required to conduct their tests as confirmation, or is it optional? If no medical record is available the enrolling site will conduct a laboratory test as confirmationprotocol text updated 2. For primary outcome "… will be measured with three questions, using their composite scores". By composite do you mean the sum of their scores? A composite score is calculated as the average of the three responses 3. For the qualitative evaluation, it indicates that "… we will elaborate on the RE-AIM quantitative findings with a brief interview for a small number of patients and ED providers." Please provide an idea of a small number (at least 5 of each according to paragraph 2 below this one). Will interviews only include participants who had a follow-up ED visit? Yes, each participating Center will conduct at least five provider and five patient interviews with participants who had an ED VOE visit post enrollment.

In the response the investigators report that they will enroll people with VOC within 30 days of enrollment but the protocol says within 90 days. If feasible it would be better to limit recall to the last 30 days. The protocol is correct at 90 days. Due to small patient population, 30 days would not provide a reasonable potential participant poll to all investigators to meet enrollment goals.

Investigators should consider excluding participants who already have IPP that may have been used at the visit prior to enrollment.. While the study sites will track who already has an IPP (prior to enrollment) and who does not, excluding those with an existing IPP would be detrimental to enrollment at some sites. Please note, having a preenrollment IPP in place does not imply the patient has had an ED visit with that IPP in place or that it was accessible to the patient and their ED care provider during any pre-enrollment ED visit

Plans for study operations: organization of clinical centers, governing committees, data coordinating center Plans for training clinical center personnel: good clinical practice, research goals, protocol details and implementation, and study operations Plans for data acquisition: data management, quality assurance, and ultimate transfer of data to NHLBI

Systematic literature review and assessment of patient-reported outcome instruments in sickle cell disease. Health and quality of life outcomes

ASCQ-Me Quality of Care (QOC) Measure. n

Quality of care in sickle cell disease: Cross-sectional study and development of a measure for adults reporting on ambulatory and emergency department care

Adult sickle cell quality-of-life measurement information system (ASCQ-Me): conceptual model based on review of the literature and formative research. The Clinical journal of pain

Comparison of Adult and Adolescent Quality of Ambulatory and Emergency Care in Sickle Cell Disease: Ascq-Me and the SHIP-HU Study

1. For recruitment (page 12 "required patient-related strategies") the first bullet is "enrollment of patients in clinic or hospital." I assume that you would only enroll patients if they had an ED visit within the last 90 days. Are you requiring the last ED visit to be at one of the participating ED's, or could it have been at an external ED? If you are allowing it to be an external ED, are you capturing this information? ED visits must be at participating EDs. See Eligibility Criteria -Patient Participants bullet 5 "At least one VOE visit to the participating site's ED in the past 90 days from enrollment" 2. Page 16, interventions two paragraph above "implementation strategies", "The hematologist or SCD provider will review… The IPP will be reviewed every 6 months by the SCD provider and updated as needed." Given there is a 1-year follow-up, then it would only need to be updated once in the study, at 6 months, correct? Correct. Updated to say "6 months post-enrollment"

Data and safety monitoring plan: including adverse event notification, medical monitor, and stopping guidelines Subject population: including appropriate representation of minorities, women, and children

Approve Approve with Changes*

Other materials *Please describe the changes deemed necessary for approval:1. The inclusion criteria "access to a cellular/mobile smart phone with unlimited data plan and free text messages (87)" seems unnecessarily restrictive. While I could not find statistics on what percentage of smart phone users have unlimited data plans, I would image that it would be low. If this criterion was eliminated, the potential "cost" of participation would need to be included in the informed consent form. removed the requirement for an unlimited data plan and free text messaging -new sentence reads "access to a cellular/mobile smart phone with access to text messaging" We added the potential costs incurred to the patient if they do not have unlimited data/free text messaging to the content form.