key: cord-0956212-1pvg3apl authors: Titov, Nickolai; Dear, Blake F; Nielssen, Olav; Wootton, Bethany; Kayrouz, Rony; Karin, Eyal; Genest, Ben; Bennett-Levy, James; Purtell, Carol; Bezuidenhout, Greg; Tan, Rheza; Minissale, Casey; Thadhani, Priti; Webb, Nick; Willcock, Simon; Andersson, Gerhard; Hadjistavropoulos, Heather D; Mohr, David C; Kavanagh, David J; Cross, Shane; Staples, Lauren G title: User characteristics and outcomes from a national digital mental health service: an observational study of registrants of the Australian MindSpot Clinic date: 2020-10-19 journal: Lancet Digit Health DOI: 10.1016/s2589-7500(20)30224-7 sha: 72bb1f2ed06681ddc49dfd3a1efe697c987fb91a doc_id: 956212 cord_uid: 1pvg3apl BACKGROUND: Interest is growing in digital and telehealth delivery of mental health services, but data are scarce on outcomes in routine care. The federally funded Australian MindSpot Clinic provides online and telephone psychological assessment and treatment services to Australian adults. We aimed to summarise demographic characteristics and treatment outcomes of patients registered with MindSpot over the first 7 years of clinic operation. METHODS: We used an observational design to review all patients who registered for assessment with the MindSpot Clinic between Jan 1, 2013, and Dec 31, 2019. We descriptively analysed the demographics, service preferences, and baseline symptoms of patients. Among patients enrolled in a digital treatment course, we evaluated scales of depression (Patient Health Questionnaire-9 [PHQ-9]) and anxiety (Generalized Anxiety Disorder 7-Item Scale [GAD-7]), as primary measures of treatment outcome, from the screening assessment to post-treatment and a 3 month follow-up. The Kessler Psychological Distress 10-Item Plus Scale was also used to assess changes in general distress and disability, and course satisfaction was measured post-treatment. OUTCOMES: A total of 121 652 screening assessments were started, of which 96 018 (78·9%) were completed. The mean age of patients was 35·7 years (SD 13·8) and 88 702 (72·9%) were women. Based on available assessment data, 36 866 (34·5%) of 106 811 participants had never previously spoken to a health professional about their symptoms, and most people self-reported symptoms of anxiety (88 879 [81·9%] of 108 494) or depression (78 803 [72·6%] of 108 494), either alone or in combination, at baseline. 21 745 patients started treatment in a therapist-guided online course, of whom 14 503 (66·7%) completed treatment (≥four of five lessons). Key trends in service use included an increase in the proportion of people using MindSpot primarily for assessment and information, from 52·6% in 2013 to 66·7% in 2019, while the proportion primarily seeking online treatment decreased, from 42·6% in 2013 to 26·7% in 2019. Effect sizes and percentage changes were large for estimated mean scores on the PHQ-9 and GAD-7 from assessment to post-treatment (PHQ-9, Cohen's d effect size 1·40 [95% CI 1·37–1·43]; and GAD-7, 1·45 [1·42–1·47]) and the 3 month follow-up (PHQ-9, 1·36 [1·34–1·38]; and GAD-7, 1·42 [1·40–1·44]); proportions of patients with reliable symptom deterioration (score increase of ≥6 points [PHQ-9] or ≥5 points [GAD-7]) were low post-treatment (of 13 058 respondents, 184 [1·4%] had symptom deterioration on the PHQ-9 and 282 [2·2%] on the GAD-7); and patient satisfaction rates were high (12 452 [96·6%] of 12 895 respondents would recommend the course and 12 433 [96·7%] of 12 860 reported the course worthwhile). We also observed small improvements in disability following treatment as measured by days out of role. INTERPRETATION: Our findings indicate improvement in psychological symptoms and positive reception among patients receiving online mental health treatment. These results support the addition of digital services such as MindSpot as a component in contemporary national mental health systems. FUNDING: None. Mental disorders and substance use disorders are major contributors to the burden of disease in Australia 1 and worldwide, 2,3 with only a minority of those affected see king or receiving evidence-based treatments. 4, 5 Barriers to care include stigma, cost, and availability of services. 6 The COVID-19 pandemic has created additional challenges, as many traditional mental health providers stopped pro viding face-to-face service. As a result, interest is increasing in the digital delivery of psycho logical services. 7 Digital mental health services (DMHS) remotely deliver mental health information, assessments, and treatment, via the internet, telephone, or other digital channels. DMHS are already part of routine care in several coun tries, operating either as stand-alone services or in con junction with traditional face-to-face care. 8, 9 For example, the Improving Access to Psychological Therapies (IAPT) service of the National Health Service (England) provides both faceto-face and digital services to patients with anxiety or dep ression; a stepped-care approach that allows patients to move from low-intensity intervention (such as guided self-help) to high-intensity intervention (tradi tional face-to-face therapy). 9 Stepped care is not a common feature of stand-alone DMHS, in which patients often report being un willing or unable to access traditional face-to-face therapy. 10 In this paper, we report outcomes from the Australian MindSpot Clinic, which by volume of patients, is one of the world's largest publicly funded DMHS. The MindSpot project was launched in December, 2012, and is funded by the Australian Department of Health as part of the Australian Government's e-Mental Health Strategy. 11 MindSpot provides information about symptoms and local mental health services, brief psychological assess ments, and therapist-guided treatments delivered via the internet and telephone to adults with symp toms of anxiety, dep ression, or chronic pain. We have previously reported results from 12 months 12 and 30 months 10 of operations, characteristics of service users during the COVID-19 pan demic, 13 and treatment outcomes for specific populations, including Aboriginal and Torres Strait Islander (Indig enous) people 14 and people born overseas. 15 In this paper, we aimed to provide a summary of demographic charac teristics and treat ment outcomes for patients registered with MindSpot over its first 7 years of operation, including service use and symptom severity, and examined trends in these characteristics over time. This study was designed as an observational study and is reported according to STROBE guidelines. 16 We evaluated all patients who registered for assessment or treatment with the MindSpot Clinic between Jan 1, 2013, and Dec 31, 2019. Ethical approval for the collection and use of patient data was obtained from the Macquarie University Human Research Ethics Committee (Macquarie University, Sydney, NSW, Australia; approval number 5201200912) and registered on the Australian and New Zealand Clinical Trials Registry, ACTRN12613000407796. MindSpot is funded by the Australian Government as a project and recruitment is ongoing as patients continue to access the service. As MindSpot is funded by the Australian Department of Health, patients seeking assessment or treatment must complete an online registration questionnaire and meet the following eligibility criteria: Australian resident eligible for publicly funded health services (ie, Medicare-funded services); aged 18 years or older; and self-reported principal complaint of anxiety, depression, or chronic pain. Patients are also provided with the terms of use explaining that non-identifiable, aggregated data could be used for reporting and service evaluation purposes. Patients are required to consent to the terms This study describes the characteristics and treatment outcomes of a large sample of consecutive users (n=121 652) of the national Australian DMHS, MindSpot Clinic, from data collected over its first 7 years of operations. We provide information about the demographic characteristics, service preferences, symptoms, and treatment outcomes for people using this particular model of digital service. We found that clinic users represented a broad cross-section of the Australian population, and used MindSpot for a variety of reasons, with most seeking a confidential assessment rather than treatment. We also found that people who engaged in treatment achieved significant reductions in symptoms, which were sustained 3 months after treatment completion. Importantly, these findings confirm the role of DMHS in providing evidence-based assessment and treatment to large numbers of people, many of whom are not accessing other services. The present findings contribute to the evidence base for DMHS in reducing barriers to care, and confirm the utility of DMHS as an important component of contemporary mental health systems. of use, either online or by telephone, before proceeding with assessment and treatment. The People register with MindSpot by creating an account and completing a screening assessment, online or by telephone. The screening assessment includes questions on demographic and service use information, and symptoms and current stressors. Participants are also asked about suicidal thoughts and plans. Those who disclose suicidal plans or intent and who can subsequently be contacted by telephone are administered a structured risk assessment aligned with the New South Wales Government best practice guidelines, 17 and safety plans are developed for all users to assist them to stay safe while seeking treatment or in the event of an increase in symptoms during treatment. 18 Those unable to be contacted are referred to local police for a welfare check. People who continue to express suicidal intent are referred to local mental health services or emergency services, depending on the urgency of the situation. However, patients with suicidal thoughts can also continue to access MindSpot services if they agree to a safety plan. MindSpot operates under compre hensive internal and external oversight and reporting that includes clinical, organis ational, and infor mation technology governance frameworks. The clinical governance frameworks align with Australian national standards for mental health services and include policies, systems, and protocols for identifying patients or others at risk, their management, clinical escalation in the event of increased risk, and training and supervision of staff. People who do not complete an assessment are sent information about managing symptoms, contact details for crisis services, and are invited to contact MindSpot. People who complete the assessment are invited to discuss their results with a therapist by tele phone (appendix p 1), who provides tailored advice over the appointment of approxi mately 25 min. An assess ment report that identifies clinically significant symptoms and includes information about how to access mental health services (including those offered by MindSpot) or other services, is sent by the therapist to the patient and, if requested by the patient, to a nominated health pro fessional, usually a general practitioner. Information is also provided about evidence-based techniques for self-managing symp toms. Participants who complete an assessment and elect for a MindSpot digital treatment course are then enrolled, unless they are considered ineligible for digital treatment by the therapist because their clinical presentation suggests the need for compre hensive or urgent face-to-face assessment. Those cases are sup ported to access specialist services. MindSpot delivers seven digital treatment courses, which were developed and validated in a series of randomised controlled trials at the Macquarie University online research clinic, the eCentreClinic. Four of the treatment courses are based on transdiagnostic principles recognising that people often experience symp toms of anxiety and depressive disorders simul taneously, and that similar psychological skills are used to treat these symptoms. The four transdiagnostic courses offered by the MindSpot Clinic are Mood Mechanic (for individuals aged 18-25 years), the Wellbeing Course (26-65 years), Wellbeing Plus (>65 years), and the Indigenous Wellbeing Course (for Aboriginal and Torres Strait Islander people). 14, [19] [20] [21] [22] These four interventions comprise evidencebased psychological treatment components, including psycho education about mediators and moderators of symptoms, cognitive therapy, behavioural activation, graded exposure, sleep training, communication and inter personal skills, problem solving, and relapse prevention. 19, 20 MindSpot also offers disorder-specific courses for obses sive compulsive disorder, post-traumatic stress disorder, and chronic pain. Patients can choose a treatment course based on symptoms and demo graphic characteristics, and via telephone consul tation with a MindSpot therapist. All courses consist of five lessons delivered over 8 weeks. Each lesson comprises a series of slides that presents the principles of psychological treatment for the target symp toms via text and images, based on an instructional design that accommodates both didactic and case-based learning. 20 Course completion is defined as completion of four or more lessons. Courses are delivered online with regular support initiated from the therapist once a week, either via telephone, secure email, or both. The therapist is also available at any time throughout the course. The approximate amount of therapist time per patient per course ranges from around 1·5 h to 3 h. 10 Therapist time includes all contact with patients, pre paration time for each patient including reading and responding to messages, and administration For the eCentreClinic see www.ecentreclinic.org For the Australian Department of Health national standards for digital mental health services see https://www. safetyandquality.gov.au/ standards/national-safety-andquality-digital-mental-healthstandards See Online for appendix and super vision time during treatment and during follow-up. Course materials are available online, although around 10% of people elect to receive materials via a printed workbook, sent by postal mail. In addition to the therapist-delivered treatment courses, a 6 month trial of telephone-based counselling was conducted in 2018, and a self-guided version of the Wellbeing Course was introduced in 2019, the results of which will be reported elsewhere. Standardised and validated symptom questionnaires are administered to patients at the screening assessment and throughout treatment. For the purposes of this study, treatment out comes on the Patient Health Questionnaire-9 (PHQ-9), Generalized Anxiety Disorder 7-Item Scale (GAD-7), and Kessler Psychological Distress 10-Item Plus Scale (K-10+) were analysed as treatment outcomes. The PHQ-9 consists of nine items measuring symptoms of major depressive disorder according to criteria of the Diagnostic and Statistical Manual of Mental Disorders, 5th edition. 23 Scores range from 0 to 27, with a score of 10 or more indicating a diagnosis of depression. The GAD-7 consists of seven items and is sensitive to the presence of generalised anxiety disorder, social phobia, and panic disorder. 24 Scores range from 0 to 21, with a score of 8 or more indicating the probable presence of an anxiety disorder. 25 The K-10+ was used as a secondary outcome measure to assess general psychological distress and disability. The first ten items comprise the Kessler Psychological Distress 10-Item Scale (K-10), with scores ranging from 10 to 50 and scores of 21 or more associated with the presence of anxiety and depressive disorders. 26 The K-10+ contains four additional questions used to assess the functional effect of the psychological distress. 27 In the current analysis, we used two of the additional questions to assess the number of full and part days a person had been out of role (unable to do usual duties and activities) in the past month. We also report the quantifiable K-10 score. Patients are admin istered the PHQ-9 and GAD-7 at the screening assessment, once a week during treatment (days 1, 8, 15, 22, 29, 36 , and 43), post-treatment (day 50), and at a 3 month follow-up (day 162). Patients complete the K-10+ at the screening assessment, the start of treatment (day 1), midtreatment (day 22), post-treatment (day 50), and the 3 month followup (day 162). Patients also complete a satisfaction questionnaire post-treatment. The satis faction questions we report on are: "Would you recommend this course to others?" and "Was it worth your time doing this course?" All questionnaires are delivered online and patients have 3 weeks to complete the post-treatment and follow-up questionnaires before they are considered closed. We did descriptive analyses of demographics, service preferences, and baseline symptoms for the total sample and for each year. For categorical variables, χ² analyses of linear-by-linear associations were used to examine trends with time. ANOVA was used to examine the significance of changes to continuous variables with time. χ² values represent changes in categorical variables over time, and F-values from ANOVA represent significant differences in dependent variables, with years as the independent variable. Generalised estimating equation (GEE) models with Wald's χ² as the test for significance were used to examine changes in symptom measures from assessment to post-treatment and the 3 month follow-up. 28 Con sis tent with the principles of intention-totreat analy ses, we imputed missing data for all patients starting treatment, using separate GEE models that assumed data were missing at random, and adjusted for baseline symptoms and lesson completion. 29 An unstructured working correlation matrix and maximum likelihood estimation were used, and gamma distribution with a log link response scale was specified to address positive skewness in depen dent variable distributions. We calculated the clinical significance of change in PHQ-9, GAD-7, and K-10 measures using per centage change in symptoms from baseline 30 and within-group Cohen's d effect sizes, based on the esti mated marginal means derived from GEE modelling at the screening assessment, post-treatment, and the 3 month follow-up. Reliable recovery was calculated as the proportion of patients whose scores were higher than the clinical cutoffs of primary measures (PHQ-9 ≥10 or GAD-7 ≥8) at assess ment and lower than the cutoffs post-treatment, with evidence of reliable change. Reliable change was defined as a change of at least 6 points on the PHQ-9 and at least 5 points on the GAD-7. 9,31 Reliable deter ioration in patients who completed treatment was defined as a score increase of at least 6 points on the PHQ-9 and at least 5 points on the GAD-7 post-treatment. 11 Data were analysed with SPSS (version 26.0). A significance level of 0·05 was used for all tests, with the Bonferroni correction applied for multiple comparisons. There was no funding source for this study. During the first 7 years of clinic operation from Jan 1, 2013, to Dec 31, 2019, a total of 121 652 online screening assessments were started, of which 96 018 (78·9%) were completed (figure). The number of people starting an assessment at MindSpot increased consistently from 2013 to 2016, and subsequently plateaued at around 20 000 per annum as directed by funding contracts. A breakdown of completed assessments by year is available in the appendix (p 2). Demographic characteristics of the total sample and by year, representing all those who started the screening assessment, are shown in table 1. The mean age of patients in the total sample was 35·7 years (SD 13·8) and 88 702 (72·9%) were women. During the 7 years of clinic operation, small but significant changes were observed in the age, sex, Indigenous status, employment, education, and mar ital status of people initiating an assessment. For age, we observed a slight increase in the proportion of people aged 18-35 years with time, and less change in the proportion aged 55 years and older (appendix p 3). The proportion of women fluctuated between 71·6% and 74·3%, and the proportion of people married decreased from 42·9% in 2013 to 37·0% in 2019. The proportion of patients born in Australia remained around 78·0%, while the proportion identifying as Aboriginal or Torres Strait Islander increased from 2·1% in 2013 to 4·4% in 2019. We observed some change in employment status over time, particularly in the proportion of students (11·1% in 2013 to 16·0% in 2019), and a concurrent increase in the proportion of people with a university degree (38·3% to 40·8%). The proportion of patients living outside capital cities remained relatively stable (38·7% for the total sample of 2013-18). Proportions of patients from each state and territory are shown in the appendix (p 4). Almost a third of respondents were from New South Wales. Reported psychological symptoms and stressors at the time of the screening assessment by year are shown in table 2. In 7 years of clinic operation, significant fluctuations were observed in symptoms. Mean scores at assessment (baseline) on the PHQ-9 decreased from 15·6 (SD 6·1) in 2013 to 14·5 (6·2) in 2019, with a con current decrease in the proportion of people self-reporting current difficulties with depression during that period. Mean baseline scores on the GAD-7 remained close to the mean for the whole period (12·5 [5·2]; with the exception of 12·9 in the first year), although the pro portion reporting anxiety or worry increased over the 7 years. Mean baseline K-10 scores decreased slightly, from 32·2 (7·5) in 2013 to 31·3 (7·6) in 2019. Based on a series of questions specifically about suicidal thoughts, intentions, and plans, the proportion of people reporting thoughts relating to suicide fluctuated between 29·9% and 34·7%, while the proportion reporting both suicidal thoughts and current intent or a plan increased from 2·4% in 2013 to 3·5% in 2019. Signi ficant changes over time were also observed in reported psycho social stressors. The proportion of people repor ting relationship difficulties increased, while the pro portions of people reporting vocational, physical health, or financial difficulties decreased (table 2) . Service use and preferences by year are reported in table 3. Significant changes were observed in the main reported purpose of using MindSpot among patients during the first 7 years of clinic operation. From 2013 to 2019, the proportion of people using MindSpot primarily for assessment and infor mation increased from 52·6% in 2013 to 66·7% in 2019, while the prop ortion primarily seeking online treatment decreased, from 42·6% in 2013 to 26·7% in 2019. Table 3 also reports the reasons participants gave for using an online service rather than a face-to-face service. Since intro duction of the question in 2015, around a third of respondents consistently reported convenience and (absence of) cost, and another third reported privacy and anonymity as their main reason. Over 7 years, 27·7% to 37·6% patients reported that they had never previously seen a mental health professional. 45·9% to 48·7% patients reported speaking to a general practitioner about their mental Generalized Anxiety Disorder 7-Item Scale Kessler Psychological Distress 10-Item Scale Data are mean (SD) or n/N (%), where numerators are the number of positive responses and denominators are the number of patients who provided an answer to that question. *Questions introduced July, 2017. †Questions regarding the duration of symptoms (anxiety or depression >12 months) were introduced Sept, 2013; for both, the denominator is the number of people who reported current relevant symptoms and provided a response to the duration question. ‡Missing data for the last quarter of 2014 due to system changes. GEE analyses showed significant overall symptom reductions in PHQ-9 (Wald's χ²=29 432·8, p<0·0001), GAD-7 (Wald's χ²=27 731·1, p<0·0001), and K-10 (Wald's χ²=33 261·8, p<0·0001). Pairwise comparisons showed that scores on all measures decreased significantly from assessment to post-treatment and from assessment to follow-up (all p<0·0001). Analyses of the clinical significance of treatment outcomes by year revealed consistent results, with symptom reductions post-treatment for all years on all measures (appendix pp 5-7). This study described the demographic characteristics, service preferences, and symptoms of more than 120 000 users of a national DMHS, collected during 7 years of clinic operations. Users of the service repre sented a broad cross-section of the Australian popu lation, many of whom were seeking a confidential assessment rather than treatment. Those who did engage in treat ment achieved significant reductions in symptoms that were sustained for up to 3 months. The results confirm the efficacy and efficiency of MindSpot in provi ding evidence-based assessment and treatment to large numbers of people, Data are n/N (%), where numerators are the number of positive responses and denominators are the number of patients who provided an answer to that question. *Missing data from July to Dec, 2017, due to system changes. †Data available from July, 2015. ‡Question introduced April, 2015; missing data for 2018 due to system changes. Table 3 : Mental health service preferences and use at assessment many of whom are not accessing other services. Our findings contribute to the evidence in support of DMHS within contemporary mental health systems. Consistent with reports before 2018, 11, 12 MindSpot has continued to serve a broad and geographically dispersed cross-section of the Australian population. Some changes in demographics and symp toms have occurred with time, including an increase in the proportion of young adult users, an increase in the proportion identifying as Aboriginal or Torres Strait Islander, and an increase in people reporting anxiety. A key observation was the increase in proportions of people reporting a primary purpose for contacting MindSpot was to receive an assessment rather than treatment. Many patients reported to therapists that a confidential assessment was the only intervention required at the time of consultation. This finding suggests that a dis cussion with a therapist about the nature of symptoms and treatment options is valued by many people, and can serve as a brief therapeutic intervention in itself. The data also raise important questions about engagement and attrition in digital and traditional mental health services, and whether all patients accessing a service can be assumed to be treatment-seeking. 8 These results confirm our view that DMHS should align with patient-centred models of care, and offer a range of services, including education, assessment, triage, support to access urgent help for people in crisis, and referral, as well as providing evidence-based treatment. With regard to treatment outcomes, the overall magnitude of clinical improvements obtained across the MindSpot treatment courses remain consistently high, with greater than 50% symptom reductions in anxiety and depression post-treatment, which were sustained for up to 3 months. Outcomes compare favourably with bench marks relating to substantial clinical improvement, low rates of deterioration, and high patient satisfaction with DMHS in other countries, 32 including when offered in primary care, 33 and via other initiatives for large-scale implementation of psychological treatment in Australia 34 and the UK. 9 This study has several limitations. We report on characteristics and outcomes of patients registering for assessment or treatment with MindSpot, which restricted our sample to a small proportion of visitors to the MindSpot website (>500 000 per year), and limits the generalisability of our results. We also acknowledge the issue of missing responses, which is a limitation of many studies, particularly those reporting outcomes obtained in routine care, in which patients are receiving a service rather than participating under controlled trial con ditions. The absence of a control group also means that we are unable to account for natural remission or the effect of missing data. However, this limitation was mitigated by the weekly collection of symptom scores during treatment and by conservative statistical modelling, 30 and we found no indication of systematic bias in trends over time due to missing data. A post-hoc analysis did find some evidence that young patients and those with severe symptoms are not necessarily continuing to or completing treatment (appendix p 8), which might affect the generalisability of our results. Generally, we found that several key demographic factors, such as the proportion of people born overseas, distribution by states and territories, Indigenous status (Aboriginal and Torres Strait Islander), and proportion living in rural or remote regions, closely matched national statistics. 12 However, we acknowledge that other factors might be under-represented or over-represented in our sample. For example, the proportion of men contacting MindSpot was always less than 30%, an underrepresentation consistent with reports that men are less likely to seek help for anxiety and depression from traditional mental health services, despite having higher rates of suicide than women. 35, 36 The question of how to engage men in both traditional mental health services and DMHS remains important and might require new service models. Despite these limitations, our results show that a high-volume digital mental health service can be successfully implemented as part of routine care. The main stren gths of this study are the analyses of comprehensive data on a large consecutive sample, combined with the regular measurement of symptoms to monitor treatment effects. Furthermore, treatment results over 7 years match those reported in earlier papers, confirming the robust nature of the digitised clinic procedures and clinical effects. As of 2020, MindSpot has been operating for more than 7 years. In that time, the delivery of health care, including some forms of mental health care, via digital technology has become increasingly acceptable. Services such as MindSpot have shown that digital deli very of care increases accessibility and convenience for patients and can reduce other barriers to care such as stigma. Other key learnings from MindSpot are that DMHS could have an important role in contem porary mental health care, not only by providing treat ment, but also by providing infor mation and assessment services to diverse groups of people that often under-utilise traditional health services, including Indigenous Australians, and people living in rural and remote regions. 8, 14 We maintain that DMHS are not a panacea and should not replace existing services, but instead can complement those services by reducing barriers and delivering evidence-based care to large numbers of patients in an efficient and cost-effective way. 8, 37 People who do not respond to DMHS can then be supported to seek more intensive treatment, consistent with a stepped-care approach. 9 An important feature of DMHS is the potential for systematic measurement of progress via treatment and outcomes, which is rarely implemented with existing service models. 9 By providing services to large numbers of people and routinely collecting and reporting data about user characteristics and clinical outcomes, DMHS are not only providing valuable benchmarking data, but are having a growing influence on the planning of mental health systems across an increasing number of countries. The routine collection and reporting of user data, with the exception of the UK's IAPT model, is not typical of publicly funded psychological services. Thus, such reporting by DMHS is not only increasing unde rstanding among policy makers on the relative strengths and limitations of different service models, but is likely to lead to increased expectations from funders and policy makers for similar reporting from traditional services. In the long term, this influence might lead to policy and funding decisions based more on evidence than traditional practice, but in the short term this will require change in the culture and operations of services that do not routinely collect or report these kind of data. Developing, delivering, and evaluating DMHS is challenging, requiring complex procedures and ongoing evaluation in the context of ever-changing technology and a rapidly evolving governance and regulatory envi ronment. Despite the challenges, we no longer need to question whether DMHS will become part of the frame work of mental health services. The new question is, how will this integration occur and how do we best integrate DMHS with existing face-toface services? Based on the preferences of many patients for more easily accessible, confidential mental health care, we believe that a need will be ongoing for stand-alone services, which provide the option of assessment and treatment and are not always linked with an existing provider. Ideally, existing mental health services should receive support to deliver both face-toface and digital mental health care, and we strongly recommend engagement during the development and implementation of these services with patients and other stakeholders, including policy makers and funders, to ensure that services are not only effective but also acceptable. Mental health professionals could then be trained and equipped to use digital tools with their own clients, to improve both quality of care and collection of treatment outcome data. Without such training and support, patients are unlikely to receive consistently high-quality care, and funders are unlikely to receive data on clinical outcomes to guide service or programme improvements. The MindSpot project has become one of the leading providers globally of DMHS as part of routine care and has deli vered mental health services with proven effectiveness to a large number of Australians in its first 7 years. The consistency of results provides support for the adoption of this model of care within the national mental health system, particularly in the present context of increased consumer acceptance of digital and telephone health-care services. Nonetheless, we maintain that the role of DMHS is to provide consumers and referrers with an additional choice of service model. The prevalence and burden of mental and substance use disorders in Australia: findings from the Global Burden of Disease Study Addressing mental health needs: an integral part of COVID-19 response Global, regional, and national incidence, prevalence, and years lived with disability for 328 diseases and injuries for 195 countries, 1990-2016: a systematic analysis for the Global Burden of Disease Study Estimating treatment rates for mental disorders in Australia Has increased provision of treatment reduced the prevalence of common mental disorders? Review of the evidence from four countries Barriers to mental health treatment: results from the National Comorbidity Survey Replication Advantages and limitations of internetbased interventions for common mental disorders From research to practice: ten lessons in delivering digital mental health services Transparency about the outcomes of mental health services (IAPT approach): an analysis of public data The first 30 months of the MindSpot Clinic: evaluation of a national e-mental health service against project objectives Australian Government Department of Health and Ageing MindSpot Clinic: an accessible, efficient and effective online treatment service for anxiety and depression Rapid report: early demand, profiles and concerns of mental health users during the coronavirus (COVID-19) pandemic A comparison of Indigenous and non-Indigenous users of MindSpot: an Australian digital mental health service A comparison of the characteristics and treatment outcomes of migrant and Australian-born users of a national digital mental health service The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies Mental health triage policy Procedures for risk management and a review of crisis referrals from the MindSpot Clinic, a national service for the remote assessment and treatment of anxiety and depression Transdiagnostic versus disorder-specific and clinician-guided versus self-guided internet-delivered treatment for generalized anxiety disorder and comorbid disorders: a randomized controlled trial Disorder-specific versus transdiagnostic and clinician-guided versus self-guided treatment for major depressive disorder and comorbid anxiety disorders: a randomised controlled trial Internet-delivered treatment for older adults with anxiety and depression: implementation of the Wellbeing Plus Course in routine clinical care and comparison with research trial outcomes Internet-delivered treatment for young adults with anxiety and depression: evaluation in routine clinical care and comparison with research trial outcomes The PHQ-9: validity of a brief depression severity measure A brief measure for assessing generalised anxiety disorder: the GAD-7 Anxiety disorders in primary care: prevalence, impairment, comorbidity, and detection Short screening scales to monitor population prevalences and trends in non-specific psychological distress Primary mental health care minimum data set. Scoring the Kessler-10 plus To GEE or not to GEE: comparing population average and mixed models for estimating the associations between neighborhood risk factors and health Wish you were here": examining characteristics, outcomes, and statistical solutions for missing cases in web-based psychotherapeutic trials Measurement of symptom change following web-based psychotherapy: statistical characteristics and analytical methods for measuring and interpreting change The first 30 months of the MindSpot Clinic: evaluation of a national e-mental health service against project objectives ICBT in routine care: a descriptive analysis of successful clinics in five countries Evaluation of The Practitioner Online Referral and Treatment Service (PORTS): the first 18 months of a state-wide digital service for adults with anxiety, depression, or substance use problems Australia's Better Access initiative: an evaluation The Australian version of IAPT: clinical outcomes of the multi-site cohort study of NewAccess Why are Australia's suicide rates returning to the hundred-year average, despite suicide prevention initiatives? Reframing the problem from the perspective of Durkheim The cost-effectiveness of the online MindSpot Clinic for the treatment of depression and anxiety in Australia The MindSpot Clinic is a project funded by the Australian Government. The authors gratefully acknowledge the patients for allowing the use of their data and the efforts of staff at Macquarie University (Access Macquarie, MQ Health, MindSpot Clinic, and eCentreClinic) in launching and operating the Clinic. All authors contributed equally to the manuscript. NT and BFD are authors and developers of the treatment courses used at the MindSpot Clinic but derive no personal or financial benefit. BW coauthored the Obsessive Compulsive Disorder Course used at the MindSpot Clinic with NT and BFD but derives no personal or financial benefit. NT serves as Chair of an Expert Advisory Group appointed to support the Australian Commission for Safety and Quality in Health Care to develop a national standards and certification framework for digital mental health services. All other authors declare no competing interests. Access to de-identified data might be provided on reasonable request. Requests are subject to the establishment of appropriate data governance, and the approval of an independent and recognised human research ethics committee. Requests must be made in writing to Dr Lauren Staples, MindSpot Clinic, Macquarie University, Sydney, NSW 2109, Australia (lauren.staples@mq.edu.au).