key: cord-0850748-vly3cx78
authors: Zhu, Yonghan; Wang, Rui; Pu, Chengyan
title: “I am chatbot, your virtual mental health adviser.” What drives citizens’ satisfaction and continuance intention toward mental health chatbots during the COVID-19 pandemic? An empirical study in China
date: 2022-03-30
journal: Digit Health
DOI: 10.1177/20552076221090031
sha: c8420250fe117d539621df12464b911cdaf1554e
doc_id: 850748
cord_uid: vly3cx78

INTRODUCTION: In order to address the psychological problems during the COVID-19 pandemic, mental health chatbots have been extensively used by public sectors. According to Theory of Consumption Values, this paper proposed an analytical framework to investigate the determinants behind users’ satisfaction and continuance intention toward mental health chatbots. METHODS: The empirical study was conducted through an online survey, facilitated by the use of questionnaire posted on the WeChat platform. Seven-point Likert scale and closed-ended questions were employed. Totally 371 valid samples were collected. The research data was tested via the partial least squares structural equation modeling. Gender, age, and income were included as control variables. RESULTS: Analysis of samples collected from 371 Chinese users suggested that personalization (functional value), enjoyment (emotional value), learning (epistemic value), and the condition of the COVID-19 pandemic (conditional value) have positive influences on user satisfaction and continuance intention, but such effects were weak. The findings also revealed that user satisfaction has weakly positive impact on continuance intention. However, voice interaction (functional value) was an insignificant predictor of user satisfaction and continuance intention. DISCUSSION: This study developed a critical perspective on the role of Theory of Consumption Values in the context of mental health chatbot usage, while Theory of Consumption Value has been increasingly employed to explain the use of AI-based public services. Thus, this research devotes to the enhancement of theoretical frameworks regarding the usage of mental health chatbots.

The ongoing coronavirus pandemic 2019 (COVID- 19) has caused millions of deaths and infected hundreds of millions of people worldwide, in turn leading to a plethora of mental disorders. 1 Government agencies in many countries have used mental health chatbots to provide virtual therapies and services. Although the use of mental health chatbots is widespread, our knowledge about what influences citizens' continuance intention to use chatbots is limited. In addition, our knowledge about the determinants of citizens' user satisfaction with mental health chatbots is limited. Such knowledge gaps significantly prevent the implementation of mental health chatbots during the COVID-19 pandemic.

The main purposes of this research are thus twofold: (1) employing Theory of Consumption Values (TCVs) as a theoretical framework and testing whether this theory can reveal the determinants behind user satisfaction and continuance intention during the COVID-19 pandemic; and (2) examining precisely the relationship between user satisfaction and continuance intention during the COVID-19 pandemic. These objectives are noteworthy for several reasons. Theoretically speaking, with the wide implementation of AI-based chatbots, it is necessary to extend the theoretical frameworks to explain user satisfaction and continuance intention. 2 However, prior research on the adoption of AI-based chatbots has mainly relied on Technology Acceptance Model (TAM). 3 Currently, the explanatory power of TCV has been widely demonstrated in the field of business, but this theory has rarely been employed to explain user behaviors of AI-based chatbots, especially mental health chatbots. Thus, this research may offer a critical insight into TCV by demonstrating whether it can be used to explain user satisfaction and continuance intention of mental health chatbots during the pandemic. Moreover, although many studies have revealed a positive relationship between user satisfaction and continuance intention when people use information systems (ISs), 4, 5 there is a lack of theoretically driven hypothesisbased research on this relationship in the public health emergency context. Therefore, this research might extend the existing literature on the relationship between user satisfaction and continuance intention in the IS area. Practically speaking, although chatbots are extensively used in the digital era, people's continuance intention to use chatbots remains low. 6 Expectation-Confirmation Theory (ECT), IS Continuance Theory (ISCT), and Technology Continuance Theory (TCT) have argued that people will be reluctant to continue using ISs if they feel dissatisfied with it. 7, 8 Thus, a deeper understanding of the antecedents of chatbot satisfaction and continuance intention can help government agencies and chatbot engineers develop chatbot systems and ultimately improve the adoption rate of mental health chatbots.

Based on TCV, this research developed a research model to examine the determinants behind citizens' user satisfaction and continuance intention. Furthermore, the relationship between user satisfaction and continuance intention to use chatbots was tested in this study.

Mental health chatbots. Chatbots, also known as virtual agents or intelligent agents, are generally defined as computer systems that use AI-based technologies to mimic humanlike behaviors and offer a task-oriented framework with evolving dialog, making it capable of participating in conversations. [9] [10] [11] A conversation between humans and chatbots is triggered by user input. The contextual awareness technologies allow chatbots to provide feedback based on receiving users' messages. 12 Mental health is considered as one of the areas that will benefit greatly from chatbot services. 11, 13 Currently, mental health chatbots are increasingly being used for therapeutic and training objectives, such as suicide prevention 14 and cognitive-behavioral therapy (Ail, 2018). In some cases, virtual agents, such as HARR-E and Wysa, are also tailored to certain groups. 11 Compared with manned hotlines, mental health chatbots are more accessible and flexible. 15 It is believed that these advantages can offer great support to the existing mental healthcare system during the pandemic. For instance, mental health chatbots can simultaneously provide therapies and services for many more people than a manned hotline, 16 in turn improving the effectiveness of service delivery. In addition, mental health chatbots are devoted to addressing the shortage of human resources, especially mental health advisers, during the COVID-19 pandemic. 1 Given that the mental healthcare systems could benefit greatly from mental health chatbots, government agencies in many countries such as Singapore, America, and China have deployed virtual agents to deliver 24-h psychological assistance during the COVID-19 pandemic. 16, 17 Many previous studies have focused on the effectiveness and acceptability of mental health chatbots. For example, Fitzpatrick et al. 18 proved that a chatbot called "Woebot" could effectively help college students with depression. Cameron et al. 19 identified some limitations of a mental health agent named "iHepr" and gave suggestions to improve its functions. However, few studies have investigated the determinant factors behind users' continuance intention to use mental health chatbots during the COVID-19 pandemic.

During the COVID-19-related lockdown, the government agencies in China then collaborated with their AI service contractors to launch a number of mental health chatbots to offer psychological assistance. For instance, a mental health chatbot named Psybot, which can listen, analyze, and talk, was employed by the local government in Hangzhou to help people express their true feelings and ease their moods. On encountering an emergency case, it sends out early warnings and asks professional human consultants to step in. 20 Another chatbot called Xiaoi, as deployed by the local government in Hubei Province. With its emotional computing and empathy module, Xiaoi could more clearly analyze citizens' problems and provide a warm-hearted interaction. 21 This study selected a mental health chatbot (Xiaolv) as the research target.

User satisfaction and continuance intention. User satisfaction is a central concept in behavioral research. 4, 22 Expectation Disconfirmation Theory is an influential theory that explains the formation of user satisfaction by comparing expectations and post-experience belief in services. 23 According to this theory, individuals often set expectations before experiencing a service. 24 After experiencing this service, a post-experience belief is generated. 25 When comparing expectations and post-experience beliefs, individuals can feel a sense of disconfirmation. 26 If post-experience belief is better than expectation, the sense of disconfirmation will be positive, and individuals will feel satisfied. 27 In contrast, if post-experience belief is worse than expected, the sense of disconfirmation will be negative, and individuals will be dissatisfied. 28 Thus, user satisfaction typically refers to the fulfillment response that results from comparing a service's perceived expectation of performance. 22 In the field of ISs, user satisfaction has been widely employed by researchers and practitioners to predict people's continuance intention. 8 The notion of continuance intention is different from the concept of acceptance intention: the former emphasizes user loyalty, while the latter highlights initial acceptance. 7 According to ECT, ISCT, and TCT, loyalty and long-term use lead to the success of IS services. 8, 29, 30 In this sense, continuance intention should be regarded as a key factor in IS success. 31 Thus, when IS technologies were being used, researchers and practitioners tried to investigate why people continued to adopt them. This was the case for e-learning, 32 online banking, 33 online shopping, 34 and online games. 31 User satisfaction is often regarded as an important predictor that stimulates continuance intention to use IS product or service. 29 Influential theories such as ECT, ISCT, and TCT ascertain that continuance intention is determined primarily by user satisfaction, which means that people who have higher levels of satisfaction with ISs are more likely to stay loyal longer. 7, 35 Generally, if individuals feel satisfied after using an IS product or service, they will believe that this product or service can also fulfill their expectations next time as well, and thus have greater motivation to continuously use it. 36 Theory of Consumption Values. TCV is a theory that reveals how people judge and make decisions to adopt a product or service. 37 TCV has been widely used to explain people's decision-making behavior in economics, business, and sociology. 38 According to TCV, products or services potentially contain multiple consumption values, including functional value, emotional value, epistemic value, conditional value, and social value. 39 Consumers attach different values to different products or services, which in return affects their intention to use. 38 Specifically, functional value is defined as the value acquired through the possession of utilitarian or physical attributes. 39 It is closely linked to practical factors such as reliability, effectiveness, and price. 38 In most cases, functional value is an important driver of consumers' decisions. 37, 40, 41 Emotional value is the utility derived from an alternative capacity to evoke affective feelings. 42 According to Sheth et al. (1991) , products or services are frequently related to emotional responses, such as esthetics and feelings. These emotions and affections often influence customers' motivation to use a product or service. Social value is defined as the utility derived from an alternative's relationship with specific groups (Yao et al., 2016). A product or service may represent a specific social group, customers can enhance a sense of belonging to that group through the consumption of such product or service. 43 Thus, people's decisions to use a product can be influenced by social value. Epistemic value is the utility of a product or service to arouse curiosity, or satisfy a desire for knowledge. 39 A product or service may be selected because people feel curious. 42 Conditional value is the utility acquired from a specific situation involved in customer choice. 40 When people are involved in a specific event or situation, they might value the use of a product or service because it can help them solve problems. As a result, people tend to use the product or service in a specific event or situation. 39, 44 For instance, people usually choose to use umbrella when it is raining. Conditional value arises suddenly, and consumers benefit from it temporarily. 45 Compared with other value components in TCV, conditional value more highlights the influence of condition on people's decisions.

The original TCV is used only to explain customers' choice motivations. Recently, this theory has presented a wider view wherein consumption values also influence user satisfaction. 26 Consumption values arise before or during the purchase process, whereas user satisfaction is created at the post-purchase stage. 38 In this sense, consumption values may be the determinants of user satisfaction.

According to TCV, the relative consumption values vary with changing context. 41 Thus, the value components in this research were contextualized based on the characteristics of mental health chatbots. According to Zhu et al., 26 personalization and voice interaction are important functions of AI-based machines. As for mental health chatbots, personalization can help offer more appropriate therapies and thus improve the effectiveness of human-chatbot interaction. Voice interaction can give chatbots more humanlike factors and make the human-chatbot interaction more natural. Thus, the functional values of mental health chatbots in this study are contextualized as personalization and voice interaction. As many previous studies have argued, enjoyment is a significant element of humanchatbot interaction. 10 The effective interaction between users and mental health chatbots induces a feeling of enjoyment, which in turn helps address psychological problems such as anxiety and stress. 46 Therefore, emotional value is contextualized as enjoyment. In addition, learning is another important element of human-computer interaction. 40, 41 Curiosity is one of our basic attributes and we are nearly oblivious of its pervasiveness in our lives. 47 Curiosity is also a component of subjective well-being. 48 As a significant notion in positive psychology, subjective well-being emphasizes that only individuals can assess their levels of well-being in any condition, including those who live with mental health conditions. 49 Thus, people can feel curious even if they encounter mental disorders. As a new AI-based technology, mental health chatbots can easily arouse curiosity and stimulate the desire to learn. Once individuals are motived to learn, they are more likely to continuously interact with chatbots to acquire knowledge. Meanwhile, through the learning process-that is, human-chatbot interaction-individuals can satisfy their desire for knowledge about mental health chatbots. As a result, user satisfaction will improve. Thereby, we employ learning to represent the epistemic value. People found themselves suddenly involved in a special condition (e.g. the national lockdown, overtaxed manned hotlines, and serious infection) due to the outbreak of the COVID-19 pandemic. Such a condition endows mental health chatbots with great utility, making them a valuable and available approach to offer psychological assistance. 26 In other words, the value of mental health chatbots can be better expressed and perceived by people during the COVID-19 pandemic. Thus, we contextualized the conditional value as the condition of the COVID-19 pandemic. However, social value is excluded in this study because a person is free of social bonds when interacting with a mental health chatbot.

Hypotheses development. Based on TCV, the functional values of mental health chatbots in this research are contextualized as personalization and voice interaction. Personalization refers to the ability of the system to provide personalized services that match users' unique characteristics and needs. 34, 50 Currently, the achievements of machine learning and data mining technology allow chatbots to deeply analyze users' historical data. Based on historical data, chatbots can offer services that may meet users' demands better. 51 According to TCV, functional values are essential in forming user satisfaction 26 and continuance intention. 41 This point has been supported by many empirical studies on AI-based applications. 52, 53 In this study, personalization as a functional value can identify specific psychological problems accurately and offering personalized therapies or services. Thus, personalization may greatly help those who are with psychological issues during the COVID-19 pandemic. Based on the argument above, we formulated the following hypotheses:

H1a:Personalization is positively related to citizens' user satisfaction with mental health chatbots. H1b:Personalization is positively related to citizens' continuance intention to use mental health chatbots.

Voice interaction is another function value of AI-based chatbots. It refers to the communication between human and chatbots through voice. 26 Because voice is commonly used by a person to interact with others in daily life, when chatbots use voice to communicate with people, people tend to feel natural and friendly. 54 As a result, the overall user experience of chatbots can be improved. 55 In this sense, the important role of voice interaction is to bring a feeling of natural and friendly. As some previous studies have proved, voice interaction may be a key variable that affects people's user satisfaction 56 and continuance intention 40 when they interact with AI-driven applications. During the COVID-19 pandemic, voice interaction may help the interaction between users and chatbots become more natural, which in return improves the overall interaction quality and might ultimately enhance user satisfaction and continuance intention. Therefore, we proposed the following hypotheses:

H2a:Voice interaction is positively related to citizens' user satisfaction with mental health chatbots. H2b:Voice interaction is positively related to citizens' continuance intention to use mental health chatbots.

In this research, we expected that the emotional value of mental health chatbots (enjoyment) shows positive influences on user satisfaction as well as continuance intention. According to prior studies, a feeling of enjoyment is important to user satisfaction with AI-based chatbots. 2, 57 As to mental health chatbots, when users interact with them, a feeling of enjoyment can contribute to addressing psychological issues such as anxiety and depression, which in return improves the overall user satisfaction. 58 Thus, we hypothesize that:

H3a:Enjoyment is positively related to citizens' user satisfaction with mental health chatbots.

For many years, enjoyment was believed to have influence on the acceptance behavior of ISs. 59 A feeling of enjoyment usually generates positive feedback and thus encourage user adoption of ISs. 60, 61 Prior studies based on TCV have proved that enjoyment as emotional value can stimulate people's motivation to adopt ISs. 41, 62 Recently, Kasilingam 57 has found that citizens' motivation to adopt chatbots for online shopping can be greatly affected by a feeling of enjoyment. When people interact with a mental health chatbot, they may feel enjoyable because this chatbot provides high-quality services and therapies. Such a feeling of enjoyment can help to reduce people's stress, anxiety, and depression, 46 which in return may encourage their continuance intention to use the chatbot. Thus, we hypothesize that:

H3b:Enjoyment is positively related to citizens' continuance intention to use mental health chatbots.

As a basic element of our cognition, curiosity motivates learning and decision-making. 47 Users choose to adopt a product or service may be because they feel curious and intend to learn. 38 According to TCV, learning is an important value component that can satisfy users' curiosity. 45 Commonly, if a product or service successfully arouses people's curiosity, people will take action to learn more about it. The learning process is usually completed through the continuous use of this product or service. 63 Thus, curiosity motivates the desire to learn, and learning drives the continuance intention to use. Moreover, if people perceive that their knowledge is enriched through the use of a product or service, they tend to feel more satisfied. 63 The research by Kim et al. 64 proved that those who improve their degree of knowledge through the use of mobile shopping applications will have a higher level of user satisfaction. In addition, Lee et al. 40 found that learning can positively affect citizens' intention to reuse Web-based taxi services. Thus, we formulated the following hypotheses:

H4a:Learning is positively related to citizens' user satisfaction with mental health chatbots. H4b:Learning is positively related to citizens' continuance intention to use mental health chatbots.

According to TCV, the condition in which a product is used drives the conditional value. 38 Although a condition or event may be temporary, it can exert a great impact on user satisfaction and continuance intention. 65 The national lockdown, overtaxed manned hotlines, and serious infections triggered the special conditions of the COVID-19 pandemic. In such a condition, people tended to highly value the use of mental health chatbots because these virtual agents were available and helpful. As a result, citizens might be more likely to feel satisfied with mental health chatbots and show greater continuance intention. The research by Zhu et al. 26 noted that the condition of the COVID-19 pandemic is positively associated with user satisfaction when using mental health chatbots. Based on the arguments above, we hypothesize that:

H5a:The condition of the COVID-19 pandemic is positively related to citizens' user satisfaction with mental health chatbots. H5b:The condition of the COVID-19 pandemic is positively related to citizens' continuance intention to use mental health chatbots.

User satisfaction is considered as a key factor to form and retain continuance intention by many theories. The early theories such as ECT and ISCT hold that user satisfaction is a basic and pivotal predictor of continuance intention. 7, 25, 66 The recent theory such as TCT also ascertains that user satisfaction plays a great role in building user loyalty. 29 Based on these theories, existing literature has proved that there is a positive link between user satisfaction and continuance intention. To list a few, Kim et al. 36 argued that continuance intention is greatly influenced by user satisfaction when people use e-commerce systems. Nascimento et al. 67 proved that satisfaction has a great impact on continuance intention to adopt smartwatch. In line with these studies, we hypothesize that:

H6:User satisfaction is positively related to citizens' continuance intention to use mental health chatbots.

All the hypotheses and the research model are shown in Figure 1 .

This is a cross-sectional study conducted using data collected from Wuhan, Chongqing, and Hangzhou, during the COVID-19 pandemic in China. A complete national lockdown was implemented in China from January to March 2020. 26 This serious lockdown made millions of people experience a sudden change. It also caused a shortage of living supplies and financial distress. 68 In such a condition, many people felt anxious, depress, and sleepless.

In order to help those who are with mental health problems, local governments launched an AI-based mental health chatbot called Xiaolv. People with issues of stress, anxiety, depression, and sleep could interact with Xiaolv through Wechat, a popular mobile application in China. 26 Figure 2 shows the typical mental health care supervised by Xiaolv. At the beginning, Xiaolv welcomes users via a self-introduction. Then, users can freely "talk" to Xiaolv about their worries, emotions, and feelings through both spoken and written languages. After analyzing users' inputs, Xiaolv will organize languages to encourage and comfort users. For instance, when a user with depression input "Due to the pandemic, I have to stay at home. I am not in mod," Xiaolv comforted this person by saying "Do not be nervous my dear, no matter what happened, I will always be with you." Such a conversation could offer a feeling of warm to users, especially those who are in home isolation, and make them believe they find a good listener. Also, Xiaolv could provide other mental health supports to help users relax, such as game recommendations.

From February to April 2020, many Chinese undergraduate students worked as volunteers to fight on the front line of urban and rural communities for epidemic prevention. Community health service centers randomly assigned several families to each student volunteer. The students then provided health services for the designated families. We contacted 49 volunteer students in Wuhan, Chongqing, and Hangzhou, they helped us to send the Xiaolv chatbot's link through WeChat groups to 166 families in 43 communities. After a week, the online questionnaire was uploaded on Wechat groups and potential respondents were requested to complete it.

Before the formal survey, several experts in the area of ISs were invited to give suggestions. In addition, we also conducted a pilot test with 23 volunteers. According to their revision advice, we modified the questionnaire, making it contain three parts. In the first part, we used three questions to identify those who had adopt Xiaolv: "could you please tell us the name of the chatbot?," "have you interacted with the chatbot?" and "can this chatbot speak to you?." Then, we collected demographic information of the respondents. Third, we measured consumption values, user satisfaction, and continuance intention through a set of items. As a response, we totally collected 445 samples. Then, we deleted those responses with missing data and those samples who indicated that they never interact with Xiaolv. Finally, 371 responses were considered as valid. The demographic information, user frequency, and duration are shown in Table 1 .

All measurement items used in this paper were adapted from relevant studies. Measurement items for personalization were adapted according to Roy et al. 69 Three items were used to measure voice interaction based on the research by Zhu et al. 26 Measurements for enjoyment were modified from Lee et al. 40 To measure learning, three items were used according to Teng. 41 Items measuring the condition of the COVID-19 pandemic were adapted from Omigie et al.. 65 User satisfaction was measured according to the research by Li and Fang. 70 Finally, continuance intention was measured based on three items from Ashfaq et al.. 2 Each item in this study was evaluated by using a seven-point Likert scale, with anchors ranging from 1 (highly disagree) to 7 (highly agree). All the measurement items were shown in Appendix. In order to make the potential respondents understand our questions, we translated the questionnaire from English to Chinese. As Brislin 71 has argued, translation is necessary in the formulation of questionnaires for cross-cultural research. In this research, we checked the back-translation according to the approach suggested by Brislin 71 and Werner and Campbell 72 : (1) simple sentences, (2) repetition of nouns rather than use of pronouns, (3) avoiding English passive tense, (4) avoiding metaphor and colloquialisms. 

Several commonly controlled variables such as age, gender, income, and education were considered as control variables in this research. Chen and Mitomo 73 believed that these specific variables might have impact on the empirical results. Thus, we included them as control variables and did not test their influences in this research.

In this study, we employed the partial least squares structural equation modeling (PLS-SEM) to examine the 11 hypotheses. According to Kasilingam, 57 PLS-SEM has the ability to compare two or more groups by specifying a permutation-based analysis of variance approach. Thus, it is widely used for hypothesis-testing analysis.

The measurement model was assessed through reliability, convergent validity, and discriminant validity. According to Table 2 , the Cronbach's alpha values for all variables are higher than 0.70, suggesting that the measurement is reliable. 74 In addition, the composite reliabilities of all the cases are higher than the recommended value of 0.70. 26 Moreover, the factor loadings of all items are greater than the desirable level of 0.70 and reported as significant. 75 The average variance extracted (AVE) ranges between 0.628 and 0.728, exceeding the recommended value of 0.5. 76 Thus, the measurement model meets the conditions of reliability and convergent validity.

In order to test discriminant validity, we used crossfactor loadings and Fornell-Larcker criterion. Table 3 shows the cross-factor loadings of all variables, proving that they loaded highest on their respective construct. 26 According to Table 4 , none of the intercorrelations of the constructs are higher than the square root of the AVE for the constructs, indicating that the condition of discriminant validity is met by the measurement model. 75 Moreover, there is an adequate model fit when the value of SRMR is less than 0.08. 57 In this research, SRMR is 0.07.

In order to check multicollinearity, we examined the variance inflation factor (VIF). Table 2 shows that all VIF values are lower than the recommended value of 5.0, 26 indicating that the research model is free from multicollinearity.

The proposed hypothesis and path coefficients were examined by applying bootstrapping procedure with 2000 subsamples and two-tailed test with a significance of 0.05 in Smart PLS 3.3. According to Figure 3 , user satisfaction (β = 0.207, t = 2.547, p < 0.01) and continuance intention (β = 0.199, t = 2.811, p < 0.01) were positively influenced by personalization, offering support to H1a and H1b. Enjoyment is positively related to user satisfaction (β = 0.225, t = 2.471, p < 0.05) and continuance intention (β = 0.152, t = 2.232, p < 0.05), offering support to H3a and H3b. In addition, the influences of learning on user satisfaction (β = 0.185, t = 2.037, p < 0.05) and continuance intention (β = 0.227, t = 2.470, p < 0.05) offer support to H4a and H4b, respectively. Similarly, the condition of the COVID-19 pandemic is positively correlated with both user satisfaction (β = 0.281, t = 2.905, p < 0.01) and continuance intention (β = 0.235, t = 3.063, p < 0.01), supporting H5a and H5b. User satisfaction also shows positive impact on continuance intention (β = 0.197, t = 2.150, p < 0.05). Thus, H6 is supported. However, voice interaction shows insignificant effect on both user satisfaction (β = 0.048, t = 1.078, p > 0.05) and continuance intention (β = 0.011, t = 0.329, p > 0.05), H2a and H2b should be rejected. Overall, 9 out of the 11 hypotheses are supported, as shown in Table 5 . According to Figure 3 , there were weak betas (<0.30) but high R 2 values (>0.67) in the research model (Seo & Bernsen, 2016) . This may be because the determinant factors were highly interrelated (shown in Table 4 ). While these interrelated variables were significantly correlated with the dependent variables on an individual level, together they might compete to explain the same variance.

AI-based mental health chatbots have been extensively used to address the COVID-19 outbreak-related mental disorders across the world. This study investigated the determinant factors behind citizens' user satisfaction and continuance intention when interacting with a mental health chatbot. In addition, it further explored the relationship between user satisfaction and continuance intention.

Specifically, voice interaction failed to bring greater satisfaction and stronger continuance intention to users. This result is consistent with the research by Zhu et al.. 26 Moreover, although personalization showed positive influences on user satisfaction and continuance intention, the effects were not as strong as the research by Chen et al. 37 Similarly, enjoyment showed weakly significant influences on user satisfaction and continuance intention. This result is in line with some previous studies. 40, 77 Meanwhile, learning was positively related to people's user satisfaction and continuance intention, but the relationship was not strong. This result is different to some prior research 26,41 that found a great impact of learning on both user satisfaction and user loyalty. The condition of the COVID-19 pandemic also had weakly significant influences on user satisfaction and continuance intention. This is consistent with the research by Zhu et al. 26 Finally, the positive relationship between user satisfaction and continuance intention was weakly supported in this research. This result is in line with the research by Ashfaq et al. 2 (weakly supported) but is different to the study by Kim et al. 5 (strongly supported).

Thus, although the positive influences of personalization, enjoyment, learning, and the condition of the COVID-19 pandemic were successfully demonstrated in this research, their effects were all rather weak. One appropriate explanation for this is that these value components of TCV frame the therapy as something to be consumed or sold to consumers, but this might be an incorrect way to consider mental health service.

As mental health service in this research was free of charge public service that offered warm, personable, and reflective care, it cannot be handled from a consumer's perspective with TCV. Thereby, although the determinants of TCV could explain user satisfaction and continuance intention, they failed to exert strong influences. Another potential explanation is that we failed to find the most appropriate value components to predict user satisfaction and continuance intention toward mental health chatbots. TCV allows researchers to contextualize value components according to the specific product 

Although value components were not strongly related to user satisfaction and continuance intention, the findings in this research can make contribution to the strengthening of theory regarding the acceptance and use of chatbots. First, this research is one of the early attempts to investigate chatbot users' satisfaction and continuance intention based on TCV. Currently, the wide use of mental health chatbots creates a demand to extend the theoretical frameworks to predict user satisfaction and continuance intention. While the existing studies have predominately used TAM, 3 this study provides a new perspective by employing TCV. Second, this study also paves the way to future studies. Through the contextualized theorizing process, we identified five value components to predict user satisfaction and continuance intention toward mental health chatbots. As the results shown, voice interaction failed to play a significant role, the influences of personalization, enjoyment, learning, and the condition of the COVID-19 pandemic were weakly significant. These findings offer empirical evidences to future studies when choosing value components.

For practices, the findings in this research had the potential to contribute to user satisfaction and continuance intention because it developed end-user awareness on what factors should be considered for better performance of mental health chatbot services. First, our findings are straightforward: the personalized service is the predictor of user satisfaction and continuance intention. Accordingly, in order to make users keep using mental health chatbots, chatbot developers and designers should ensure that their products can offer personalized, relevant, and effective mental services. The design process should take into account the chatbot's capacity to understand user's demands and provide appropriate therapy.

Second, if people feel enjoyable when interacting with a mental health chatbot, they will tend to keep using this chatbot. Recently, with the evolutionary development of entertainment technologies, gamification techniques and features have been increasingly used in human-computer interaction to generate a feeling of enjoyment. 78 In the context of mental health chatbot usage, chatbot developers and designers can added more gamification features into mental health chatbots to make the human-chatbot interaction more pleasant and enjoyable. For example, these chatbots may recommend more funny games to users. It is believed that gamification elements have potential to offer a feeling of enjoyment and ultimately improve user satisfaction and continuance intention.

Moreover, if people can enrich their knowledge and satisfy their desire to learn when using a mental health chatbot, they will tend to feel satisfied and continuously adopt it. Thus, chatbot developers need to ensure that people can satisfy their desire to learn through the human-chatbot interaction. For instance, chatbot 

This research is not without limitations. First, as mentioned above, one of the most obvious limitations in this study is the use of TCV because the value components of TCV fail to exert strongly positive influences on user satisfaction and continuance intention. Second, there are problems regarding the generalizability of the findings, as the research samples were only collected from Wuhan, Hangzhou, and Chongqing. Future studies should conduct international investigation to validate the findings of this research. Third, we only chose a mental health chatbot as a research target, the results may be influenced by the advantages or shortages of the chatbot. Thus, researchers should consider more research targets in the future. Fourth, most of the correlations between factors were over 0.7, indicating that these determinant factors may share commonalities. This may influence the findings. Finally, this study considered demographic factors as control variables. Future studies should aim to test the potential moderating effect of demographic factors on continuance intention.

The wide use of mental health chatbots during the COVID-19 pandemic provides an opportunity to explore the determinant factors behind users' satisfaction and continuance intention toward these virtual agents. This paper is one of the early attempts to investigate the usage of a mental health chatbot through the lens of TCV. Although the findings outlined the significance of personalization, enjoyment, learning, and the condition of the COVID-19 pandemic, their effects were all rather weak. Also, the positive relationship between user satisfaction and continuance intention was weakly supported. On the other hand, this research found that voice interaction is an insignificant predictor of user satisfaction and continuance intention. All these findings developed a critical perspective on the role of TCV in the context of mental health chatbot usage, while this theory has started to be increasingly used in the context of AI-based public service delivery. Based on the findings, several managerial implications for chatbot developers and designers were suggested to improve user satisfaction and continuance intention of mental health chatbots.

Although there were limitations in this research, it might make considerable contributions to both existing and future studies. First, it filled in our knowledge gaps by identifying four determinants that influenced user satisfaction and continuance intention of mental health chatbots during the COVID-19 pandemic. Moreover, it developed a critical perspective on TCV that prior studies could not acknowledge. In addition, it provided empirical evidence to enhance our understanding on the relationship between user satisfaction and continuance intention. Future studies can benefit from the findings in this research when investigating the usage of mental health chatbots. 

Potential role of artificial intelligence to address the COVID-19 outbreak-related mental health issues in India

chatbot: modeling the determinants of users' satisfaction and continuance intention of AI-powered service agents

Adoption of AI-based chatbots for hospitality and tourism

User experience, satisfaction, and continual usage intention on IT

The effects of service interactivity on the satisfaction and the loyalty of smartphone users

How business can begin using chatbots the right way

Understanding information systems continuance: an expectation-confirmation model

Extension of technology continuance theory (TCT) with task technology fit (TTF) in the context of Internet banking user continuance intention

Chatbots: are they really useful?

Communicability of traditional interfaces VS chatbots in healthcare and smart home domains

Chatbots and conversational agents in mental health: a review of the psychiatric landscape

Designing voice user interfaces: principles of conversational experiences

Acceptability of artificial intelligence (AI)-led chatbot services in healthcare: a mixed-methods study

Virtual therapists help veterans open up about PTSD

Telehealth for global emergencies: implications for coronavirus disease 2019 (COVID-19)

Artificial intelligence and COVID-19: a multidisciplinary approach

Online mental health services in China during the COVID-19 outbreak

Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (Woebot): a randomized controlled trial

Assessing the usability of a chatbot for mental health care

Government in Hangzhou recommends mental health chatbot 2020

The chatbot Xiaoi offers several AI-based services to support epidemic prevention

A framework for marketing management. Essex: Pearson Education Limited

Effect of expectation and disconfirmation on postexposure product evaluations: an alternative interpretation

Disconfirmation of consumer expectations through product trial

Measurement and evaluation of satisfaction processes in retail settings

It is me, chatbot: working to address the COVID-19 outbreak-related mental health issues in China. User experience, satisfaction, and influencing factors

Incorporating trust-in-technology into expectation disconfirmation theory

Unravelling the relationship between response time and user experience in mobile applications

Information technology adoption behavior life cycle: toward a Technology Continuance Theory (TCT)

Understanding Chinese users' continuance intention toward online social networks: an integrative theoretical model

Understanding continuance intention toward crowdsourcing games: a longitudinal investigation

Perceived fit and satisfaction on web learning performance: IS continuance intention and task-technology fit perspectives

The role of continuous trust in information systems continuance

The role of personalization on continuance intention in food service mobile apps

Understanding consumers' continuance intention to contribute online reviews

Trust and satisfaction, two stepping stones for successful e-commerce relationships: a longitudinal exploration

AI-based self-service technology in public service delivery: user experience and influencing factors

A study of Malaysian customers purchase motivation of halal cosmetics retail products: examining theory of consumption value and customer satisfaction

Why we buy and what we buy: a theory of consumption values

A customer value theory approach to the engagement with a brand: the case of KaoKao talk plus in Korea

Look to the future: enhancing online gamer loyalty from the perspective of the theory of consumption values

Consumer perceived value: the development of a multiple item scale

Social capital production in a virtual P3 community

Searching the fit pattern between cultural dimensions and consumption values of mobile commerce in Taiwan

Effect of consumption values on consumer satisfaction and brand commitment: investigating functional, emotional, social, and epistemic values in the running shoes market

AI-powered mental health chatbots: examining users' motivations, active communicative action and engagement after mass-shooting disasters

The psychology and neuroscience of curiosity

Assessing psychological health: the contribution of psychological strengths

A feasibility study of psychological strengths and well-being assessment in individuals living with recurrent depression

Values of silent commerce: a study using value-focused thinking approach

Conceptualization of omnichannel customer experience and its impact on shopping intention: a mixed-method approach

Factors affecting hotels adoption of mobile reservation systems: a technology-organization-environment framework

The personalization of conversational agents in health care: systematic review

Design of a smart storytelling toy based on voice interaction

A review of verbal and non-verbal human-robot interactive communication

Predicting user satisfaction with intelligent assistants

Understanding the attitude and intention to use smartphone chatbots for shopping

An overview of the features of chatbots in mental health: a scoping review

A meta-analysis of mobile commerce adoption and the moderating effect of culture

Explain the intention to use smartphones for mobile shopping

Factors influencing higher education students to adopt podcast: an empirical study

Why people use online social media brand communities. A consumption value theory perspective

Testing evolutionary theory of household consumption behavior in the case of novelty -a product characteristics approach

Do information and service quality affect perceived privacy protection, satisfaction, and loyalty? Evidence from a Chinese O2O-based mobile shopping application

Customer pre-adoption choice behavior for M-PESA mobile financial services. Extending the theory of consumption values

The antecedents and consequences of customer satisfaction for firms

Wearable technology: what explains continuance intention in smartwatches?

Modeling the effects of Wuhan's Lockdown during COVID-19

Constituents and consequences of smart customer experience in retailing

Predicting continuance intention toward mobile branded apps through satisfaction and attachment

Back-translation for cross-cultural research

Translating working through interpreters, and the problem of decentering. Handbook Cultural Anthropol

The underlying factors of the perceived usefulness of using smart wearable devices for disaster applications

Comparing attitudes towards adoption of e-government between urban users and rural users: an empirical study in Chongqing municipality, China

Public and private value in citizen participation in E-governance: evidence form a governmentsponsored green commuting platform

Evaluating structural equation models with unobservable variables and measurement error

Predicting the intentions to use chatbots for travel and tourism

Identifying relevant segments of AI applications adopters -expanding the UTAUT2's Variables

Acknowledgments: We are grateful to the editor and anonymous reviewers for their valuable comments on this research. In particular, we should appreciate professor Theo Arvanitis and editor Kaur Simran.

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.Contributorship: Yonghan Zhu wrote the original draft of the manuscript, completed revisions, participated in the conceptualization of the manuscript content, and conducted data collection and data analysis. Rui Wang completed revisions, conducted data collection, and assisted in the data analysis. Chengyan Pu conducted data collection and provided revisions on the manuscript. All authors approved of the final manuscript draft. CI3. I will recommend mental health chatbots to others.