key: cord-0729860-3ohp3xqy authors: Busquet-Duran, Xavier; Esteban-Perez, Manel; Manresa-Domínguez, Josep Maria; Moreno, Susana Martin; Leston-Lado, Lola; Torán-Monserrat, Pere title: Intra-rater reliability and feasibility of the HexCom advanced disease complexity assessment model date: 2022-05-09 journal: Aten Primaria DOI: 10.1016/j.aprim.2022.102343 sha: 2eb88a50d050693ab689d1aa1facbfd96080be8b doc_id: 729860 cord_uid: 3ohp3xqy AIM: To study the intra-rater reliability and feasibility of the HexCom complexity assessment model by analyzing internal consistency, intra-rater reliability and response time. DESIGN: Test–retest study with a selection of 11 clinical situations that cover the full scope of situations assessed by the HexCom model and which are responded to individually. LOCATION: Home care, primary care, hospital and sociosanitary care. Two specialized palliative home care teams (PADES). PARTICIPANTS: A total of 20 professionals comprising 10 experts in palliative home care (PADES) and 10 professionals from general palliative care (primary care, hospital and sociosanitary care). These professionals came from the fields of family medicine (5), internal medicine (2), geriatrics (2), nursing (9), psychology (1) and social work (1). MAIN MEASUREMENTS: Cronbach's alpha, weighted kappa, response time. RESULTS: Cronbach's alpha of 0.91 for HexCom-Red and 0.87 for HexCom-Clin. Intra-rater reliability ranging from good to very good for HexCom-Red (kappa: 0.78–1) and from moderate to very good for HexCom-Clin (kappa: 0.58–0.91). Average response time of 0:57 for HexCom-Red and 3:80 min for HexComClin. CONCLUSIONS: HexCom-Red and HexCom-Clin are reliable tools and feasible for use by all professionals involved in both general and specialized palliative care at different levels. End-of-life circumstances differ from one person to the next, as do a person's needs and the complexity of care to be managed by professionals. 1 General palliative care is provided by the community's primary care and hospital professionals while specialized, more resource-intensive palliative care is focused on offering support to situations characterized by greater care complexity. 2 In other words, it is the level of complexity that determines whether support is required from a team of specialists. That being so, a care model for patients with advanced disease and/or at the end of life was developed in Catalonia; it is known as the Hexagon of Complexity (HexCom) and is based on the patient's needs and strengths, as well as their care environment. 3 It stems from the work of a group of experts coordinated by La Societat Catalano-Balear de Cures Pal·liatives in collaboration with La Societat Catalana de Medicina Familiar i Comunitària under the guidance of the Sociosanitary Master Plan of the Department of Health of the Government of Catalonia. 4 The model includes six domains of need: clinical, psychological/emotional, social/family, spiritual, ethical and death-related. For each domain, the level of complexity can be low, moderate or high. Various instruments have been produced from this model, including HexCom-Red, which has six items focused on facilitating management/referral between care levels, and HexCom-Clin, which has 18 items for clinical application by specialized teams. In a recent systematic review, HexCom-Clin was deemed one of the most comprehensive complexity assessment tools for palliative care on the international stage. 5 The two instruments share domains and subdomains, which means the 18 subdomains of HexCom-Clin are covered by the six areas of both tools. A Delphi study was recently published on the face validity of the model 6 and included the participation of 14 experts from both primary care and hospitals and both general and specialized palliative care. The results showed a high level of agreement, with a content-item validity rate exceeding 0.92, thus confirming that the model is considered useful for a wide range of health professionals for recording complexity in a coordinated fashion in sociosanitary practice. An earlier study by Esteban using HexCom-Red on a population of 500 patients demonstrated excellent consistency across the intensity of intervention provided by the home care support team (PADES) and primary care (kappa 0.92), as well as across intervention from PADES and hospital specialists (kappa 0.83). Consistency remained at this level and was homogenous when analyzed based on clinical typology (oncology or otherwise) and according to patient prognosis (complex chronic patients, advanced disease or end of life). 7 Similarly, studies using HexCom-Clin support the predictive, construct and discriminant validity of the model. 8, 9 It has thus been corroborated that there exists an association between complexity and place of death, greater complexity and younger patients and cancer patients, and greater complexity and patients with cancer and neurological disorders as they approached death. One fundamental aspect of the psychometric validation of the instruments in clinical use is completion time and consistency across responses (intra-rater reliability). 10 Neither of these issues has been studied for the HexCom model. Due to the clinical instability of palliative patients, who undergo clinical changes and rapid deterioration, we must often scrutinize written or filmed cases 11 and simply accept the limitations inherent to this methodology. The aim of this study is to evaluate the intra-rater reliability and feasibility of the HexCom complexity assessment model through internal consistency, intra-rater reliability and response time. Study design: Test---retest study with a selection of 11 clinical situations that cover the full scope of situations assessed by the model. These situations were created by the principal investigator (Annex) and focused on three patient typologies: cancer (three clinical situations), advanced disease due to organ failure (four), and advanced dementia (four). Complexity was assessed on an individual basis immediately after reading each case, and response time was recorded. Multicenter study: Granollers PADES and Reus PADES. Participants and setting: Study population: Two populations of currently active professionals who had prior experience with the model and were representative of final users. Group One: Experts in palliative home care working on a PADES team who had participated in prospective multicenter study that used HexCom-Clin. Participants: Four doctors (two specialized in family medicine, one internist and one geriatrician), four nurses, one psychologist and one social worker. Base: Granollers PADES. Group Two: Professionals not specialized in palliative care working in primary care, home care, hospital or sociosanitary care who had participated in the intra-rater reliability study that used HexCom-Red. Participants: Five doctors (three specialized in family medicine, one internist and one geriatrician), five nurses (two from hospital, one from primary care, and one from case management). Base: Reus PADES. Variables: Items from the HexCom-Clin and HexCom-Red instruments and completion time (in minutes and seconds, timed). Procedure: A random list was created of combinations of three non-repeating situations, which were later assigned to each of the 10 raters following the list. All 10 raters read the three cases twice over a period of 15 days. Thirty assessments were collected per group, for a total of 60 tests; the process was repeated for the retests (for a total of 120 assessments). Statistical analysis: Internal consistency was analyzed using Cronbach's alpha and intra-rater reliability was measured using the weighted kappa. The latter was interpreted according to Landis and Koch's criteria 12 : slight (<0.20), fair (0.21---0.40), moderate (0.41---0.60), substantial (0.61---0.80) and almost perfect (0.81---1.00). The significance level was p ≤ 0.05. The statistics package software Stata for Windows, version 16.0, was used for all analyses. Sample size: Considering an intraclass kappa agreement of no less than 0:42 (H0), with a significance level of 5% in a one-sided comparison, with a power of 80% and expecting a minimum rate of complexity of 50% for each item, 30 test-retest assessments were needed to determine how significant a true kappa of 0.80 (H1) was. Timeline: June---September 2019: The clinical situations were written up and the participating professionals were selected. In December 2019, the two test---retest assessments were conducted 15 days apart. Internal consistency was high for both instruments (Cronbach's alpha: 0.91 HexCom-Red; 0.87 HexCom-Clin). Intra-rater reliability (Table 1 ) ranged from good to very good for HexCom-Red (kappa: from 0.786 to 1) and from moderate to very good for HexCom-Clin (kappa: from 0.582 to 0.918). In the case of HexCom-Red, all items showed very good consistency (kappa > 0.80) except for the clinical domain, which was good (kappa: 0.78). When agreement was analyzed by domain, it was found to be good for HexCom-Clin in the psychological/emotional, social/family, ethical and death-related domains (kappa: from 0.65 to 0.73), moderate in the clinical domain (kappa: 0.58) and very good in the spiritual domain (kappa: 0.91). As for the subdomains of HexCom-Clin (Table 2) , agreement was good for most items (kappa: from 0.64 to 0.78) except for clinical---physical, psychological---emotional, social/familyemotional and -practical, ethical-decisions and -DHD and death-related grief, which were moderate (kappa: from 0.40 to 0.58). As for the resources section, agreement was moderate for most items, and good for microsystem-patient (kappa: 0.74) and very good for mesosystem-family links (kappa: 0.80) and strength-financial (kappa: 0.84). Feasibility: Mean response time in minutes and seconds was 0:57 for HexCom-Red and 3:8 for HexCom-Clin. Response time was significantly lower for the retest (Table 3 ). The results of this study demonstrate good internal consistency for both instruments, very good intra-rater reliability for HexCom-Red and moderate to good intra-rater reliability for HexCom-Clin, as well as excellent feasibility for both HexCom-Red, which took participants less than one minute to complete, and HexCom-Clin, with a response time of less than four minutes. There are some limitations inherent to the study design. Among feasibility analysis parameters, test-retest feasibility is a significant psychometric property. Due to the clinical instability of palliative patients, who undergo rapid deterioration, we must often study written or filmed cases. 11 Living through a situation in real life, watching a video of it, or reading about it on paper constitute radically different experiences. This fact represents an insurmountable Study outline: Test---retest study with a selection of 11 clinical situations that cover the full scope of situations assessed by the HexCom model and which are responded to individually. limitation of this type of study. The methodology of our study is like that used to assess the reliability of the Japanese version of Support Team Assessment Schedule (STAS-J), in which 16 nurses and one doctor working in a palliative care unit rated 10 hypothetical cases on two separate occasions. 13 The interval between assessments is another limitation of the results: we followed the system used to validate the Spanish version of the Supportive and Palliative Care Indicators tool for low-income settings (SPICT TM ), in which the retest was conducted 10 days after the test. 14 One strong point of our study is that it included professionals from all fields and levels involved in both general and specialized palliative care. Our results are comparable to those found in the literature and can be considered good when compared with other instruments commonly used in palliative care. In the case of NECPAL CCOMS-ICO©, in a study with two doctors and two nurses, Rodríguez-Calero identified moderate agreement (kappa 0.50), 15 while in the Chilean version, Troncoso described it as ranging from 0.632 and 1.0. 16 As for the Integrated Palliative care Outcome Scale (IPOS), good test-retest reliability has been reported for the subscales physical symptoms (intra-class correlation coefficient [ICC] = 0.77) and emotional symptoms (ICC = 0.72), and moderate for the subscale communication and practical issues (ICC = 0.53). 17 With respect to the Palliative Care Outcomes Collaboration Symptom Assessment Scale (PCOC SAS), Daveson found a weighted kappa of 0.6 (CI 95% 0.54---0.63) for the pain item. 18 Similarly, for the Japanese version of the STAS-J, which was also analyzed using hypothetical cases, a weighted kappa of 0.64---0.85 was reported. 13 We also discern similarities when comparing our results for response time with those found in the literature. For example, a response time of 2.3---2.4 min was described for the Supportive and Palliative Care Indicators Tool for a low-income Setting (SPICT-LIS). 19 In the case of the Palliative Care Needs Assessment Tool (PC-NAT), 20 completion time ranged from 5 to 10 min Participants took around 5 min to complete the NECPAL CCOMS-ICO©, although this figure varies considerably from one study to the next, ranging from 1.9---7.72 minutes in a study by Rodríguez-Calero 15 and 6.7 ± 4.01 minutes in a study by Troncoso. 16 Much like us, Troncoso also found that response time improved with training, decreasing from 6.7 min to 3.9 min the second time around (p ≤ 0.001). It should be noted that rater subjectivity and, especially, non-linearity, interdependence and changes over time play a 21, 22 In these conditions, professionals responded on an intuitive basis, which is not as easily activated in the context of fictitious written cases as it is in the context of real care. As Gigerenzer puts it, the response to complexity is simplicity, and a truly efficient healthcare response requires mastering the art of focusing on what's important and forgetting the rest. 23, 24 In a situation where information is lacking and uncertainty abounds, simple diagnostic methods tend to be the best option. 25 In that regard, HexCom-Red falls in line with the growing appearance of tools that allow various professionals in the system to share their vision and personalize patient care through multi-dimensional assessment within a limited time frame and which can be regularly administered by a single professional in any area of the health system. 26, 27 The internal consistency (Cronbach's Alpha) and stability of responses (intra-rater reliability) identified in our study, alongside the face validity results, 6 endorse the use of the HexCom model in clinical practice. Nevertheless, as Carvajal noted, validating an instrument is a continuous and dynamic process that acquires greater consistency as more psychometric properties are measured across different cultures, with different populations and subjects. 10 We have thus sought approval from the IDIAPJGol Clinical Research Ethics Committee for the formal study ''Validation of an advanced disease complexity assessment model (HexCom)'' (code 19/215-P), which is to be based on real patients rather than written cases and includes assessment of concurrent validation. 5 This study, however, is currently postponed due to the COVID-19 pandemic. If carrying out fieldwork becomes unfeasible, a set of videotaped role-play simulated situations would be used to test the HexCom tool. These would include the necessary variables to make those scenarios comparable to real-life situations. The analysis of videotaped performances often yields some interpretative variability on the part of the observers. Nevertheless, these allow anticipating the benefits and difficulties of implementing the instrument before it is used in real healthcare settings. In conclusion, HexCom-Red and HexCom-Clin are reliable instruments and feasible for use by all the professions involved in both general and specialized palliative care at different levels. • The criterion for access to specialized palliative care must be the care complexity of patient needs. • Of the various models designed to assess complexity, the HexCom model is one of the best rated internationally. • HexCom-Red was designed to manage and refer cases whereas HexCom-Clin is intended for clinical care and research. • It demonstrates the reliability of the model, with good results for internal consistency and intra-rater reliability. • It shows the feasibility of the model, with a response time of less than one minute for HexCom-Red and four minutes for HexCom-Clin. This study obtained partial funding from the University Institute for Primary Care Research IDIAP JGol (4R17/053) and a research grant for the principal investigator (XB). Departament de Salut. Bases conceptuals i model d'atenció per a les persones fràgils, amb cronicitat complexa (PCC) o avançada (MACA) Direcció General de Planificació en Salut del Departament de Salut White Paper on standards and norms for hospice and palliative care in Europe: Part 2 Debate: criterios de complejidad utilizados en cuidados paliativos. El hexágono de la complejidad Descripció i consens dels criteris de complexitat assistencial i nivells d'intervenció en l'atenció al final de la vida. Barcelona: Direcció General de Planificació i Avaluació A systematic review of classifications systems to determine complexity of patient care needs in palliative care Assessing face validity of the hexcom model for capturing complexity in clinical practice: a Delphi study Concordancia entre la complejidad observada desde diferentes niveles asistenciales en pacientes crónicos complejos, con enfermedad avanzada o al final de la vida mediante un modelo de abordaje de la complejidad Describing complexity in palliative home care through HexCom: a cross-sectional Trayectorias de complejidad en final de vida: un estudio multicéntrico prospectivo observacional longitudinal How is an instrument for measuring health to be validated? A critical analysis of test---retest reliability in instrument validation studies of cancer patients under palliative care: a systematic review The measurement of observer agreement for categorical data Reliability and validity of the Japanese version of the Support Team Assessment Schedule (STAS-J) Spanish adaptation and validation of the supportive & palliative care indicators tool ---SPICT-ES TM Detección de necesidad de atención paliativa en una unidad de hospitalización de agudos, Estudio piloto Adaptación y validación en Chile del instrumento de identificación de pacientes con necesidad de atención paliativa: NECPAL-CCOMS-ICO 3.1©. Atención Primaria Validity and reliability of the English and translated Chinese versions of the Integrated Palliative care Outcome Scale (IPOS) in Singapore The PCOC Symptom Assessment Scale (SAS): a valid measure for daily use at point of care and in palliative care programs Systematic adaptation of the Thai version of the supportive and palliative care indicators tool for low-income setting (SPICT-LIS). BMC Palliat Care Development of the palliative care needs assessment tool (PC-NAT) for use by multi-disciplinary health professionals Understanding complexity ---the palliative care situation as a complex adaptive system A framework for complexity in palliative care: a qualitative study with patients, family carers and professionals Simple tools for understanding risks: from innumeracy to insight Decisiones instintivas Fast, frugal, and fit: simple heuristics for paired comparison VIG-Express: consenso de un sistema de valoración multidimensional/geriátrica rápida en Cataluña Instrument per a la valoració del nivell de complexitat d'atenció de persones amb cronicitat avançada i necessitats pal·liatives (MACA): HexCom-Red We would like to thank all the study participants for sharing their time and knowledge with us. We would also like to thank the members of the Granollers and Reus PADES teams. Lastly, we would like to show our appreciation to Txus Micó for his secretarial help. The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Supplementary data associated with this article can be found, in the online version, at doi:10.1016/j. aprim.2022.102343.