key: cord-0815803-6wcv7foi authors: Messina, Irene; Loffler-Stastka, Henriette title: Psychotherapists’ perception of their clinical skills and in-session feelings in live therapy versus online therapy during the COVID-19 pandemic: a pilot study date: 2021-04-15 journal: Res Psychother DOI: 10.4081/ripppo.2021.514 sha: 9bc7d5c9bf84920d0b5161e581ae653ff76fe402 doc_id: 815803 cord_uid: 6wcv7foi During the coronavirus disease 2019 (COVID-19) pandemic, many psychotherapists who were used to seeing their patients in face-toface setting adapted to providing therapies online. In the present pilot study, we investigated therapist current experiences of online therapy compared to live therapy. Twenty-nine therapists completed Clinical Skills, Difficulties in Practice, and in-sessions feelings of Flow, Boredom and Anxiety of the Trainee Current Progress Report, giving a score for each item in two different conditions: Live Therapy and Online Therapy. Compared to Live Therapy, in Online Therapy therapists reported significantly less Clinical Skills, whereas Difficulties in Practice did not differ in the considered conditions. With regard to in-sessions feeling, therapists reported significantly lower scores of Boring and higher scores of Flow in Live Therapy compared to Online Therapy, whereas Anxiety did not differed in the considered conditions. Online therapy can be defined as 'any type of professional therapeutic interaction that makes use of the Inter-net to connect qualified mental health professionals and their clients' (Rochlen et al., 2004) . The efficacy of online therapy is supported by clinical trials in which it has been compared to waiting-list control groups (Klein & Richards, 2001; Lange et al., 2000; Lange, van de Ven, Schrieken, & Emmelkamp, 2001) and to traditional faceto-face therapies (Cohen & Kerr, 1999; Day & Schneider; 2002) , and it has been documented for a variety of clinical conditions including post-traumatic stress disorder (Poletti et al., 2020; Backhaus et al., 2012) , anxiety disorders (Poletti et al., 2020) , depression (de Bitencourt Machado et al., 2016) , obsessive-compulsive disorder (Simpson, 2009) , panic attacks and agoraphobia (Poletti et al., 2020) , bulimia nervosa (Simpson, 2009 , de Bitencourt Machado et al., 2016 and mixed diagnoses (Backhaus et al., 2012) . Moreover, a number of studies have investigated various aspects of the therapeutic relationship in online therapy, such as therapeutic alliance (Cook & Doyle, 2002; Reynolds et al., 2013) and trust (Fletcher-Tomenius & Vossler, 2009) , and have provided qualitative description of online therapy processes William et al., 2009 ). This early evidence accounts for similar features of the therapeutic relationship in online therapy and face-to-face therapy (Sucala et al., 2012; Cook & Doyle, 2002) . In the recent past of these studies, in Italy online therapy was confined to very specific situations, such as individuals living in isolated areas, 'internet-friendly' patients and therapists or therapies realized as part of clinical trials specifically proposed to test online therapy efficacy. During the coronavirus disease 2019 (COVID-19) pandemic lockdowns, instead, the use of online therapy has become usual in the Italian context. Most psychotherapists, who were used to seeing their patients in face-to-face setting, adapted to provide therapies online, via videoconferencing or via telephone. Thus, the COVID-19 pandemic is offering the opportunity to investigate features of online therapy in large scale. Among different interesting aspect of online therapy experience, we have notified that, despite available evidence accounts for a general efficacy of online therapies, most therapists continue to prefer traditional live therapy (van der Vaart et al., 2014) or they have relevant reservations about online therapy (Cipolletta & Mocellin, 2018) . These preferences suggest the existence of differences in their experience of online compared to live therapy. However, to the best of our knowledge, no study has investigated features of online therapy experience from the therapist point of view. This became the aim of the present pilot study. We started from previous investigations of psychotherapists' point of view and experiences in doing therapy that have been object of attention by the Interest Section on Therapist Training and Development of the Society for Psychotherapy Research (SPRISTAD) (Orlinsky et al., 2015; Messina et al., 2019) and its ancestor of the SPR Collaborative Research Network (CRN) Orlinsky et al., 2001) . In this context, core aspects of the therapists' experiences from different countries, professions, and theoretical orientations were identified . Among others contributions, the SPR/CRN has collected data concerning the quality of therapist involvement in doing psychotherapy, such as their perception of clinical skills and difficulties in doing therapy, as well as in-session feelings (Orlinsky, Rønnestad & Willutzki, 2010; Orlinsky et al., 1991; Orlinsky et al., 1999a) , observing their variation by therapist and across time . Moreover, variation of these dimensions have been confirmed to be important features of psychotherapists development during their training (Messina et al., 2017) . The aim of the present study is to extend these previous investigations considering how therapist experience may vary in association to live (traditional) versus online settings. Following SPRISTAD tradition, we focused in three main dimensions of healing involvement and stressful involvement in therapist experience . First, we considered therapist perception of their abilities (healing involvement) and difficulties in doing therapy (stressful involvement) in online compared to live psychotherapy sessions. Second, we were interested in therapists' in-sessions feelings. We took into consideration the experience of flow as a component of healing involvement, defined as a deep interest and engagement felt during therapy sessions, and considered an important component of healing involvement in doing therapy, and feelings of anxiety and boredom, considered components of stressful involvement in doing therapy . In our hypothesis, differ-ences in therapists experiences of healing involvement (perceived abilities and in-session feelings of flow) and stressful involvement (perceived difficulties and in-session negative feelings) may differ in live therapy compared to online therapy. A convenience sample of 29 volunteer therapists with different theoretical orientations and with experiences in doing therapy in online settings were recruited through online social networks. The general features of therapists sample are summarized in Table 1 . As shown in Table 1 , therapists' experience in online sessions became more frequent during the COVID-19 lockdown and the use of online sessions was partially maintained post lockdown period. Parts of the Trainee Current Progress Report (TCPR) were used to collect the data of the present study. The TCPR is the core longitudinal instrument of the SPRIS-TAD project (Orlinsky et al., 2015) and it has been shown to be suitable to measure changes in therapists perceptions of their experiences, with sufficient sensitivity in measuring change in therapists over the course of the time (Messina et al., 2017) . Among the various areas relevant to therapists' description and evaluation of their therapeutic practice, for the aims of the present study we considered the subscales Clinical Skills, Difficulties in Practice, in-sessions feelings of Flow, Anxiety and Boredom. A previous study has showed good to acceptable internal reliability in the TCPR subscales Clinical Skills (α=0.97), [ Difficulties in Practice (α=0.82), Anxiety (α=0.73), Boredom (α=0.73), Flow (α=0.72) (Messina et al., 2017) . The multiple item subscale for Clinical Skills includes 12 items (rated on an 11-point scale from 0='not at all' to 10='very much') that ask respondents how they perceive themselves on various aspects of technical skill (e.g., 'How much mastery do you have of the techniques and strategies involved in practicing therapy?') and relational skill (e.g., 'How effective are you at engaging patients/clients in a working alliance?'). In the present study, the internal reliability of the subscales Clinical Skills was excellent (α=0.96). The subscale Difficulties in Practice included 12 items (rated on an 11-point scale from 0='never' to 10='very often') relating to how often the therapist feel self-doubt regarding one's own ability as a therapist (e.g., 'Lacking in confidence that you can have a beneficial effect on a client'), being unable to move therapy with a client in a constructive direction (e.g., 'Bogged down with a client in a relationship that seems to go nowhere'). In the present study, the internal reliability of the subscales Difficulties in practice was excellent (α=0.92). In-session feelings were evaluated asking therapists the rating of a list of possible in-session feeling on a 6point scale (from 0='never' to 5='very often'). The subscale Flow included the items inspired, stimulated, engrossed, challenged; the subscales Anxiety included the items anxious, overwhelmed, pressured, trapped; and the sub-scale Boredom included the items absent, bored, drowsy. In the present study, the internal reliability was good for subscales Boredom (α=0.85) and Anxiety (α=0.78), whereas it was problematic in the case of subscale Flow (α=0.49) Data were collected in the post-lockdown period. After giving their informed consent, therapists completed an online form with the selected parts of the TCPR. In the same moment, therapists evaluated their current experiences in online therapy and live therapy. They reported their ratings of perceived Clinical Skills, Difficulties in Practice, and in-session feelings of Flow, Anxiety and Boredom, referred to two different conditions: Live Therapy and Online Therapy. Thus, we presented each subscale 2 times, one time referred to current experience in online therapy (for example 'Currently, doing online therapy…') and one time referred to current experience in live therapy (for example 'Currently, doing live therapy…'). Due to absence of normality in data distribution, we used permutation t-test for statistical analyses. In permutation method P-value are obtained from the sample-specific permutation distribution of that statistic, rather than from the theoretical distribution as in common parametric methods (Pesarin & Salmaso, 2010; LaFleur & Greevy, 2009) . We compared mean scores of all dependent variables (Clinical Skills, Difficulties in Practice, in-session feelings of Flow, Boredom and Anxiety) in Live Therapy and Online Therapy conditions in the same group of therapists, with variable therapist considered as random factor. In these analyses, we also included age, gender, theoretical orientation, experience and experience in online therapy as covariates. Moreover, only for explorative purposes, we also evaluated difference between Live Therapy and Online Therapy conditions in single items of Clinical Skills and Difficulties in Practice subscales. Therapists perceive differently their performance in doing therapy in live and online settings Significantly higher scores of Clinical Skills were observed in Live Therapy compared to Online Therapy (t=3.34, P<0.001). Moreover, this difference was observed in most single items of the subscale (see Table 2 for single items results). Instead, no significant results were reported in the subscale Difficulties in Practice (t=0.69, P<0.519). Exploring results related to single-items, we observed a statistically significant result for the item 'Unable to generate sufficient momentum to move therapy with a client in a constructive direction' (t= -2.04, P=0.035) and a statistical trend for the item 'Unable to have much real empathy for a client's experiences' (t= -1.83, P=0.078). In both cases, scores were lower for Live Therapy compared to Online Therapy (more difficulties in Online Therapy). With regards to in-session feelings, significant differences have been reported in the case of Flow, with more experience of flow-related feeling in Live Therapy compared to Online Therapy (t=2.83, P=0.012), and Boredom, with less experience of boredom-related feelings in Live Therapy compared to Online Therapy (t= -2.76, P=0.011), whereas no significant differences were found in the case of Anxiety (see Tables 3 and 4 for in-session feelings results). We did not find significant effects of age, gender, theoretical orientation, experience and experience in online therapy. In the present pilot study, we investigated psychotherapists' perception of online therapy compared to live therapy (or traditional therapy), with a focus on variables that According to our results, the quality of therapists' experience differed between traditional therapy and online therapy. This differences clearly emerges in therapists' perception of reduced clinical skills in online therapy compared to live therapy. For example, therapists reported different perception of effectiveness in engaging the patients in therapeutic alliance, in their feeling of being 'natural', in communication and in the management of transfer and controtransfert issues and others. Instead, in case of therapist perceptions of difficulties in psychotherapy practice differences did not reach statistical significance. Taken together, these results suggest that even if therapists did not perceived clear difficulties or errors in online sessions, at the same time they have a general impression of not practice therapy as the best they can. May this differences in therapists' perceived performance influence psychotherapy effectiveness? On the basis of our data, it is not possible to draw conclusion regarding how much the observed difference in therapists' perception of their therapeutic skills may influence psychotherapy outcome. From psychotherapy effectiveness research, however, it is clear that psychotherapy outcome is importantly predicted by the person of the therapist (from 5% in controlled studies and to 17% in health care studies) (Baldwin & Imel, 2013) . The effect sizes for therapist effects are 0.13-0.74, the relapse rates of severe psychiatric illnesses are between 25 and 100% depending on the ther-apist and the frequency of therapy discontinuations is related to the therapist variable (Baldwin & Imel, 2013) . According to this evidence, therapists' performance may have an influence. In the present study, we also observed that the quality of psychotherapists' experience in online and live therapy differed reported in-session feelings, with decreased flow and increased boredom in online therapy. In this case, the focus was on therapists' feelings, which may appear less directly connected to psychotherapy outcomes. Nevertheless, previous studies have clearly related therapists' insession feelings with psychotherapy outcomes, with faster symptom reduction in presence of high flow and low boredom (Heinonen et al., 2014; Heinonen et al., 2012) . Thus, we can hypothesize that also differences between in-session feelings in online and live therapy may influence therapy outcomes producing differences in term of effectiveness (note, however, that the reliability of Flow subscale was not satisfactory in the present study). In sum, the results of the present study, together with therapists' variable studies, bring into question the effectiveness of online therapy compared to more traditional therapies. On the other hand, early comparisons of traditional face-to-face and online therapies did not report differences in terms of efficacy (Cohen & Kerr, 1999; Day & Schneider; 2002) . This inconsistence can be explained with several hypothesis. A hypothesis is that therapists' perception may reflect a real decline in therapists performance in online therapy, but this decline is not significant enough to influence psychotherapy outcome (at least as measured in clinical trials). An alternative hypothesis is that therapists' performance decline has significant ef- Second, clinical studies involved specific groups of therapists, such as therapists with specific trainings in online therapy or scientific interest for online therapies, elements that may influence their experience of online therapy. With this regard, it is important to mention that a specific training in online therapy may help therapists' in cope against the difficulties that online setting implies, with a possible reduction of differences with traditional therapy. The issues touched in the present article may have relevant implications for the understanding of online therapy, enlarging the scientific debate from efficacy as reported in clinical trials to clinical effectiveness evaluated extending the investigation to other important factors, including the therapists' point of view and experiences. However, important limitation of the present study should be considered. First, the generalizability of our findings remains to be tested because of the small convenience sample of therapists. Second, we did not consider patients-related variables, such as their diagnosis, psychological impairment severity or preferences for online versus live therapy. Third, we did not control if therapists' perceptions regarded therapies that switched from live to online settings or therapies that have maintained the same modality throughout the time. Thus, the continuation of data collection is required for a stronger empirical anchoring of the hypotheses emerged in the present pilot study. Bergin and Garfield's handbook of psychotherapy and behavior change Construing a therapeutic relationship online: An analysis of videoconference sessions Online counseling: An exploratory survey of Italian psychologists' attitudes towards new ways of interaction Computer-mediated counseling: An empirical study of a new mental health treatment Working alliance in online therapy as compared to face-to-face therapy: Preliminary results A psychodynamic perspective on a systematic review of online psychotherapy for adults Trust in online therapeutic relationships: The therapist's experience. Counselling Psychology Review Therapists' professional and personal characteristics as predictors of outcome in long-term psychodynamic psychotherapy and psychoanalysis Therapists' professional and personal characteristics as predictors of outcome in short-and long-term psychotherapy A brief Internet-based treatment for panic disorder Introduction to permutation and resampling-based hypothesis tests Interapy. Treatment of posttraumatic stress through the Internet: a controlled trial Internet-mediated, protocol-driven treatment of psychological dysfunction Trainees' self-evaluation of their development as psychotherapists: An Italian contribution to an international collaborative study on psychotherapy training An overview of the Italian contribution to the international multisite SPRISTAD study on psychotherapy training Development of psychotherapists: Concepts, questions, and methods of a collaborative international study Towards an empirically grounded model of psychotherapy training: Four thousand therapists rate influences on their development Outcomes and impacts of psychotherapists' personal therapy: A research review How psychotherapists develop: A study of therapeutic work and professional growth Psychotherapists' assessments of their development at different career levels The SPR collaborative research program on the development of psychotherapists A collaborative study of development in psychotherapy trainees Permutation tests for complex data: theory, applications and software Telepsychotherapy: a leaflet for psychotherapists in the age of COVID-19. A review of the evidence Impact of exchanges and client-therapist alliance in online-text psychotherapy Online therapy: Review of relevant definitions, debates, and current empirical support Clinical implications: Training, supervision, and practice Professional development and personal therapy Supervision of beginning and advanced graduate students of counseling and psychotherapy The journey of the counselor and therapist: Research findings and perspectives on professional development Therapist and relationship factors influencing dropout from individual psychotherapy: A literature review Psychotherapy via videoconferencing: A review Blending online therapy into regular face-to-face therapy for depression: content, ratio and preconditions according to patients and therapists using a Delphi study Insession processes in online counselling with young people: An exploratory approach Acknowledgments: the authors would like to thank the SPRISTAD group, in particular (in alphabetical order) Armin Hartmann, Erkki Heinonen, David E. Orlinsky, Michael Helge Ronnestad, Thomas Schröder, and Ulrike Willutzki for their ongoing valuable input, fruitful discussion on the data presented here and collaborative support for the whole Society of Psychotherapy Research Interest Section on Therapist Training and Development -SPRISTAD project.