key: cord-0922339-6lms8lrt
authors: Jun, Seung-Pyo; Yoo, Hyoung Sun; Lee, Jae-Seong
title: The impact of the pandemic declaration on public awareness and behavior: Focusing on COVID-19 google searches
date: 2021-01-13
journal: Technol Forecast Soc Change
DOI: 10.1016/j.techfore.2021.120592
sha: 6ac56137d683afcc2608c7fd238d27156b914956
doc_id: 922339
cord_uid: 6lms8lrt

The unprecedented outbreaks of epidemics such as the coronavirus has caused major socio-economic changes. To analyze public risk awareness and behavior in response to the outbreak of epidemic diseases, this study focuses on RSV (Relative Search Volume) provided by Google Trends. This study uses the social big data provided by Google RSV to investigate how the WHO's pandemic declaration affected public awareness and behavior. 37 OECD countries were analyzed and clustered according to the degree of reaction to the declaration, and the United States, France and Germany were selected for comparative study. The results of this study statistically confirmed that the pandemic declaration increased public awareness and had the effect of increasing searches for information on COVID-19 by more than 20%. In addition, this rapid rise in RSV also reflected interest in the COVID-19 test and had the effect of inducing individuals to be tested, which helped identify new cases. The significance of this study is that it provided the theoretical foundation for using RSV and its implications to understand and strategically utilize public awareness and behavior in situations where the WHO and governments must launch policies in response to the outbreak of new infectious diseases such as COVID-19.

This study investigates how the declaration of the coronavirus (COVID-19) pandemic, which was the third time the World Health Organization (WHO) declared a pandemic since its establishment in 1948 (Broberg, 2020) , affected the public. Specifically, this study attempts to empirically demonstrate the ensuing human behavioral changes and impact on public awareness of COVID-19 by tracking Google search data before and after the WHO's pandemic declaration. Here, "public awareness" refers to the public's level of understanding about COVID-19 (Capano et al., 2020) . Google search data can objectively measure the public's information search behavior (Jun, 2012a; , and this study utilizes the relative search volume (hereinafter referred to as RSV) provided by Google Trends (Do et al., 2015; Husnayain et al., 2020) . This empirical analysis is critically important because changes in public awareness regarding a global pandemic is one of the key factors in diagnosing and preventing new cases and strengthening group immunity (Do et al., 2015; Hu et al., 2020; Randolph and Barreiro, 2020) . Therefore, this study analyzes how the WHO's pandemic declaration, an important event, affected public risk awareness (or perceptions) and behavior in response to the outbreak of this epidemic disease.

The key question investigated in this study is, "How did the WHO's pandemic declaration affect public awareness and behavior at the beginning of the global COVID-19 outbreak?" We aim to demonstrate how interventions such as the WHO's pandemic declaration can play an important role in strengthening public awareness as well as the responses of policy authorities in each country, by analyzing public reactions as reflected in information search activities in response to the WHO's pandemic statement. In addition, this study will illuminate how these phenomena can be observed, how their results or effects can be measured, and furthermore, how these observations and measurements can be utilized in the future.

To figure out how we ought to interpret and utilize RSV changes in the unprecedented situation of a new pandemic such as COVID-19, we need to understand the determinants and impacts of RSV changes. We need to know the determinants of RSV changes to understand the significance of RSV in the process of monitoring RSV, and we also need to understand these determinants to make decisions based on the monitored RSV changes. In addition, the analysis of determinants will also indicate which variables should serve as the basis for clustering countries for hypothesis testing. To analyze the RSV determinants, this study cross-sectionally analyzed changes in RSV in 37 OECD countries, changes which occurred before and after the declaration. To analyze the impact of RSV, we analyzed and compared the temporal changes of RSV from a longitudinal perspective over 100 days in the cases of the United States, Germany, and France, which were countries that were identified as belonging to different clusters in our determinants analysis. We thereby determined how the rise of RSV varied due to the declaration, how the increase persisted, and how it was correlated to new tests and new (confirmed) cases of This study assumed that the policy changes implemented in each country immediately after the declaration are within the scope of the same event effect. Section 2 reviews preceding studies on infectious diseases that used RSV, including studies on policy changes in each country immediately after the declaration, and reviews other recent studies related to COVID-19 that were conducted based on social data. Our research model was constructed on the premise that changes in awareness will manifest as changes in information search behavior, from the perspective of human behavior; relevant explanations on this point are provided in Section 3 along with the explanation of our research methodology. Section 3 also explains the data used in this study. Section 4 presents our research results with explanations of the findings of our analyses of RSV determinants and RSV effects. Section 5 concludes with a discussion of the results of this research and the policy implications for utilizing the results.

Risk awareness among the public is critically important to prevent the spread of infectious diseases (Guo et al., 2015) . It is possible to significantly reduce the spread of infectious diseases by pre-emptively disseminating knowledge about how widely infectious diseases are spreading and how severe the outbreaks will be and educating the public on precautionary measures (Funk et al., 2009; Wang et al., 2019) . In addition, to effectively respond to the spread of infectious diseases, it is important for people to recognize the general symptoms of infection, so that they may determine whether or not they are infected by comparing their own symptoms with those of people around them, and then take appropriate measures accordingly (Zang, 2018) . In the past, people mostly relied on mass media and word of mouth to obtain information about an infectious disease and how to prevent it and respond to it (Wu et al., 2012) . However, with the availability of the internet and the widespread use of search engines, people began to seek information more actively, and this effort is reflected in the data of their search activity. There have been many studies of behavioral analysis or forecasting using RSV in various fields, not only regarding infectious diseases, and in many such studies, RSV was understood through the lens of the human behavior processes described in theories such as the five stages of the consumer buying process or the innovation adoption model (Kotler et al., 2014; Rogers, 2003) . According to these theories, problem awareness leads to information searching and the seeking of alternatives for problem solving (Jun et al., 2014) . In the same vein, RSV regarding infectious diseases has been found to be useful for quantifying the level of people's awareness of infectious diseases (Boehm et al., 2019) . Through specific keyword analysis, it is possible to distinguish, for example, whether people want to know "what" infectious disease is spreading is and "what" kinds of symptoms indicate infection, or whether are seeking information on "how" to prevent and treat it (Cairo, 2020) . Because Google has a dominant share (92%) of the global search engine market (Statcounter, 2020) , the RSV of Google searches has been recognized to be an especially effective means of gauging people's perception of infectious diseases (Do et al., 2015; Hu et al., 2020) .

In this context, researchers have steadily used RSV to observe and monitor infectious diseases, identify awareness levels, and predict spread (Nuti et al., 2014 ). The first leading example is the study by Ginsberg et al. (2009) , who presented a model that predicts current flu levels using Google Trends, the RSV provided by Google, and demonstrated that it can predict the outbreak of influenza one to two weeks earlier than the Center for Disease Control (CDC). Pelat et al. (2009) analyzed the relationship between RSV and the incidences of three other infectious diseases, namely ILI, gastroenteritis, and chickenpox. Based on their results, they reported that RSV can be used broadly to monitor diseases other than influenza and that even applying one or two well-chosen keywords can be highly effective. In addition, it has been reported that RSV can be effectively used to monitor the spread of various other infectious diseases such as Lyme disease, dengue fever, and Zika virus (Althouse et al., 2011; Seifter et al., 2010; Strauss et al., 2020) . After the release of the Google Flu Trends (GFT) service using Google's RSV, the accuracy of these predictions was recognized in many countries (Cook et al., 2011; Dugas et al., 2012) . However, researchers have also steadily pointed out problems in the accuracy of GFT and some of the fundamental limitations of predicting infectious diseases using RSV (Butler, 2013; Lazer et al., 2014; Olson et al., 2013; Pollett et al., 2016) . Butler (2013) reported that the GFT algorithm greatly overestimated the peak flu level in 2013. Butler argued that because RSV can be affected by external effects such as the mass media, and because it is difficult to identify the reasons motivating searches, there may be limitations in using RSV to predict the level of infectious diseases. Lazer et al. (2014) also cautioned against hubris, arguing that while social big data such as RSV can be a useful tool for understanding people's behavioral patterns, the accuracy of predictions based on such data is limited, and therefore such data should be used to supplement rather than replace conventional methods. The GFT service was terminated in 2015 due to various issues, but there have been continued efforts to improve the performance of nowcasting and forecasting of infectious diseases using RSV (Lampos et al., 2015; Preis and Moat, 2014; Santillana et al., 2014) . Yang et al. (2015) used Google Trends to propose ARGO (AutoRegression with Google search data), an influenza tracking model with more accurate prediction performance than GFT. Furthermore, more recent studies have evaluated that RSV can provide higher accuracy and predictive power when used in combination with conventional monitoring methodology, stronger compared to when each is applied individually (Kandula and Shaman, 2019) . Althouse et al. (2019) used Google Trends data to confirm a strong correlation between search traffic and the number of hospitalizations. Based on these results, the authors argued that Google Search data can be useful for decision-making on medical and public health issues because it has the advantage of being updated quickly without additional costs.

Meanwhile, a number of studies have been published recently on the relationship between RSV and COVID-19, which has spread around the globe; these will be discussed in detail in the next section.

On the subject of COVID-19, many studies have used sources of social data such as news, Twitter, and Facebook, or attempted quantitative approaches using RSV, as we do in this study. The main areas of research conducted in this manner can broadly divided into two. First, there are studies that tracked changes in the public's psychology regarding the epidemic to determine the country's general response to infectious diseases. The other remaining studies can be sub-divided into studies empirically analyzing unconfirmed new symptoms, the number of new cases, or the number of hospitalizations. The contents of these preceding studies are reviewed in the following.

The proliferation of an infection disease such as COVID-19 also has a significant impact on public psychology (Katafuchi et al., 2020) . First, studies on psychological changes among the public using social big data include the following examples. Li et al. (2020b) studied the changes in the results of sentiment analyses of posts written by Weibo users before and after the WHO pandemic declaration. The results demonstrated that after the WHO's pandemic declaration, indicators of anxiety, depression, and indignation increased significantly. In addition, the study found that social risk judgment was significantly increased, and life satisfaction was significantly decreased. Hamidein et al. (2020) conducted a survey of participants who regularly checked COVID-19-related news and found that they generally felt negative emotions in response to the news but were also deriving problem-solving strategies through the news, identifying what can be done at an individual level to prevent the spread of the virus. Brodeur et al. (2020) focused on the lockdown implemented by many governments in response to COVID-19. The authors found that lockdowns may help to contain the spread of the virus but warned that it may result in substantial damage to the population's well-being. To assess the level of such damage, this study used Google Trends data to analyze the causal effects, tracking how topic search terms related to well-being changed before and after the lockdown policy. Based on this analysis, the study evaluated the impact of governments' lockdown policies on the public's mental health. Chen et al. (2020) evaluated public perception of COVID-19 through Weibo, a Chinese social media platform. This study underscored the importance of how experiences of outbreaks of similar diseases in the past affect the public's perception of the outbreak of a new infectious disease.

Meanwhile, the following studies attempted to provide empirical analyses of major issues related to COVID-19 using social data such as RSV as quantitative data. Walker et al. (2020) noted that while initial reports describing COVID-19 symptoms mainly pointed to cough, breathlessness, and fever, there were as yet unverified reports suggesting that anosmia may also be a manifestation. Their study used Google Trends to investigate the presence of this unverified symptom, by tracking whether there was an actual surge in individuals searching for information related to smell loss during the COVID-19 epidemic in several major countries. The study found strong correlations between loss of smell and increases of daily COVID-19 cases and deaths in all of the major countries. The study by Hu et al. (2020) adopted a similar perspective and used Google Trends to investigate the global public perception of COVID-19. The study analyzed the period of two months before and after January 30, the date on which the WHO announced a Public Health Emergency of International Concern (PHSIC), and claimed that the first peak was observed immediately after the WHO's PHSIC announcement. This study examined the correlation between RSV and daily confirmed patients and found statistically significant positive correlations in many countries. The authors pointed out that the reaction time was different in each country and the overall duration of public attention was short, and argued that efforts should be made to strengthen public awareness on COVID-19 at the national level and to reinforce public vigilance and sensitivity to COVID-19 (Hu et al., 2020) .

There have been various studies that demonstrated that RSV has a very significant correlation with new cases in individual countries. Husnayain et al. (2020) reported that moderate to strong correlations between Google relative search volumes (RSVs) and COVID-19 cases were found in Taipei, and provided studies using specific search terms such as face mask, handwashing, etc. Ayyoubzadeh et al. (2020) analyzed data in Iran to show that search terms such as handwashing and hand sanitizer can be significant variables in predicting COVID-19 incidence. Li et al. (2020a) analyzed internet searches conducted in Baidu and Weibo as well as Google, and argued that the COVID-19 outbreak occurred 10-14 days earlier than the peak of daily incidences in China, demonstrating the internet search data can have effective surveillance capabilities for tracking the outbreak of a new disease.

Cairo (2020) analyzed search patterns found in RSV and found that in the case of the United States, after the first confirmed case in the country, there was a brief increase in searches, most of which asked "what is/are," but after February 29, when the first deaths occurred and the number of confirmed cases in other countries increased significantly, the searches surged significantly again. Specifically, prior to March, most searches were queries for general information such as "What is coronavirus?" but beginning in March, especially after the actions of the WHO and the United States government, there was a surge in searches for "What are symptoms of coronavirus?" or for "how to" information.

As summarized above, most studies analyzing social data related to COVID-19 focused on changes in the public's emotional states such as anxiety, sadness, or stress, in response to governments' lockdown policies to stem the spread of the infectious disease. While the studies reviewed above also researched the effects of the pandemic declaration, our study differs in that we focused on changes in the public's search patterns in response to the outbreak of disease and their subsequent behavior instead of focusing on changes in the perceived emotional state of the general public. What distinguishes this study is that we explain the serial process leading from risk awareness to information searching, testing, and confirmed cases through the analysis of changes in RSV.

The WHO proclaimed a Public Health Emergency of International Concern (PHEIC) on January 30, and eventually declared the situation a pandemic on March 11, which led many countries to adopt responsive policy measures. As mentioned above, in this study we regard the policies implemented immediately after the WHO pandemic declaration as the effect of the same event and therefore we found it necessary to understand the policy changes that occurred immediately after the WHO pandemic declaration. We briefly examined the policy changes in the three countries we selected as case studies based on our longitudinal study.

On March 11, the WHO declared the outbreak to be a pandemic. By this time, the virus had spread to 110 countries (Thelocal.de, 2020) and all continents except Antarctica (Nakamura and Managi, 2020; WHO, 2020) . The WHO's definition of a pandemic "mixed severity and spread," reported Vox, and it held off calling the outbreak a pandemic because many countries at the time were reporting no spread or low spread (Piper, 2020) . In the United States, on March 11, when the WHO declared COVID-19 to be a global health pandemic, later that same day President Trump announced new restrictions on travel from Europe (Goodman, 2020) . In a prime time, Oval Office address to the nation, President Trump announced new travel restrictions to and from 26 European countries, said that the United States had taken early intense action, and commended the "dramatically fewer cases of the virus in the United States than are now present in Europe." Travel restrictions went into effect on March 13 for 26 European countries in the Schengen Area and similar restrictions entered into effect on March 16 for the United Kingdom and Ireland (Woodyard, 2020) . On the same day, President Trump declared that the coronavirus pandemic to be a national emergency. He also stated: "I don't take responsibility at all," in response to a question about the lack of available tests. Also, on March 16, the White House advised against any gatherings of more than ten people (Liptak, 2020) .

Meanwhile in Germany, the citizens of Germany were shocked when President Trump of the United States announced on March 12 the 30-day travel ban for foreigners who travelled from Schengen area states, including Germany. German politicians criticized the United States for adopting this measure without prior agreement, and complained of the exclusion of the United Kingdom from the ban (Vitzthum, 2020) . On March 13, 14 of the 16 federal states of Germany decided to close all kindergartens, schools and Kitas (daycare centers) up to April 6 (Thelocal.de, 2020). Germany's neighbors, the Czech Republic, Poland and Denmark, closed their borders (BBC_news, 2020a). On March 22, the government and the federal states prohibited gatherings of more than two persons for a minimum of two weeks and agreed to impose a social distancing rule of 1.5 meters between individuals in public spaces with the exception of family members, partners or other cohabitants. Service establishments such as restaurants and hair salons were closed on the same date (Ohms, 2020) .

In France, on March 12, President Emmanuel Macron announced on public television that all schools and universities would close on Monday, March 16, until further notice (Cuthbertson, 2020b) . The next day, Prime Minister É douard Philippe banned gatherings of more than 100 people, not including public transport. The following day, the prime minister ordered the closure of all non-essential public places, including restaurants, cafes, cinemas, and nightclubs, effective from that midnight (BBC_News, 2020b). On March 16, President Macron declared a compulsory lock-down in homes to go into effect at noon on March 17, for a duration of 15 days. This lock-down measure was extended twice and ended only on May 11 (Cuthbertson, 2020a) .

Synthesizing the preceding studies, we can conclude that the WHO's pandemic declaration increased the public's social risk awareness and judgment, and using the information they already had or newly acquired (for example, through searches), the public adopted problem-solving strategies that could be performed at an individual level (Hamidein et al., 2020; Husnayain et al., 2020; Li et al., 2020b) . It has also been shown that the above process is closely related to public awareness (Cairo, 2020; Chen et al., 2020; Hu et al., 2020) . The studies also demonstrated that search data can help identify new symptoms or may have high correlation with the number of new cases, indicating that such data is useful for quantitatively analyzing major issues related to COVID-19 (Ayyoubzadeh et al., 2020; Li et al., 2020a) . We also confirmed that this series of processes is in line with the human behavior process described in the consumer buying process or innovation adoption model (Jun et al., 2014; Kotler et al., 2014; Rogers, 2003) .

Based on these existing studies and theories, we judged that once the WHO pandemic declaration raised public awareness, this would arouse public interest and increase RSV as people seek information to understand COVID-19 and find countermeasures. In addition, we hypothesized that the increased RSV would contribute to identifying new confirmed cases by motivating people with symptoms to take tests. Fig. 1 shows the research model of this study, which outlines the process explained above according to the public (or consumer) behavior model.

The key research hypotheses we sought to verify in Fig. 1 are as follows. The hypotheses stated here can be explained as characteristic phenomena that emerge in the initial stages of a new pandemic such as COVID-19.

H1) The intervention of the WHO in the form of the pandemic declaration will increase public awareness. H2) Increased public awareness will result in an increase in the discovery of confirmed cases.

The two hypotheses were both posited from the perspective of the impact of the COVID-19 pandemic declaration on awareness (or RSV). Hypothesis 1 is about the effects of the WHO COVID-19 pandemic declaration on awareness, as explained in Fig. 1 , and about the changes in behavior resulting from this shift in awareness. To test this hypothesis, we must first conduct a determinant factor analysis to identify the factors that can affect changes in RSV. This is partly because of the practical need to understand these determinant factors to monitor and strategically utilize RSV, but also because the COVID-19 declaration may be a determinant factor for awareness, or RSV, and each country exhibited varying RSV responses to the COVID-19 events (Hu et al., 2020) . To select the countries in which to test our hypotheses regarding the impacts of WHO pandemic declaration, we must take the determinants into consideration. Therefore, this study was divided into two major areas of research, centering on the information search (RSV) shown in Fig. 1 , analyzing the determinants of RSV changes and the impacts of RSV. Fig. 2 presents our research methodology in more detail.

In the RSV determinants analysis in Fig. 2 , which was designed to investigate the question "What determinants affected RSV other than the WHO declaration?" we performed case analyses while controlling for the effect of the pandemic declaration intervention. We looked for determinants that affected the rate of change in RSV in the event week (from March 11 to March 18) compared to the previous week (from March 4 to March 10) in OECD countries. 1 Based on previous comparative studies of countries regarding COVID-19, the median age, share of the elderly population, GDP, and hospital beds were considered as control variables for identifying the determinants (Chakraborty and Ghosh, 2020; Wang et al., 2020; Wu et al., 2020) . Total COVID-19 tests and new cases were considered as COVID-19 situation variables. Here, the new cases variable was also analyzed as the rate of change in the event week compared to the previous week. As for the total number of tests, we analyzed the cumulative data as a country-specific constant value (square root), like a control variable. 2 Certain limitations in the data on COVID-19 tests forced us to examine the two situational variables using different types of data, respectively in the form of the rate of change and a constant value. On March 11, when the WHO COVID-19 declaration occurred, there was a relatively large number of countries in which COVID-19 incidence was still in the early stages, and therefore daily or weekly test statistics were available in many cases. For this reason, for the variable of the tests by country, we used data on cumulative tests up to March 18, which was more easily available, rather than using the rate of change. Also, for new cases, the ratio was directly analyzed, but for cumulative tests, we converted the data to the square root. This was because as we analyzed the relatively few cases of the 37 OECD countries, we found that the distribution of the cumulative test values deviated greatly from normality. We used regression analysis as our research method, as shown in Fig. 2 , and for our main method of inference, we adopted the best subset selection method, which is superior even to conventional regression analysis based on the method of forward or backward stepwise selection (James et al., 2013) . The problem with the conventional method of forward stepwise selection is that it fails to exclude already included variables, while in backward stepwise selection, the problem is that variables that have already been excluded are not included again. The mixed (or hybrid) method solves some of the problems of forward and backward stepwise selection but fails to completely eradicate the same problems (James et al., 2013) . By contrast, the best subset selection method fits all possible subsets and identifies the best subset by comparing criteria such as Cp, AICC, and BIC. To overcome the limitations of regression analysis based on the existing least squares method, methods such as Lasso based on statistical learning have been proposed (Hastie et al., 2017; James et al., 2013) . We also conducted Lasso analysis for this study, and the results obtained from the Lasso analysis is presented below briefly, for the sake of comparison. In this study, the results of the Best Subset Selection and Lasso methods were similar, but in the Lasso method all variables were required to have no missing values and this restriction made the method applicable to fewer cases, and furthermore, the data available was somewhat insufficient for performing statistical learning. Based on these considerations, we chose to proceed with our explanation based mainly on the results from the Best 1 We analyzed changes in weekly data for a week (March 11 -18) starting from March 11 when the event occurred. to offset the seasonal changes of daily data. 2 In the case of the cumulative number of tests, we used the data as of March 18 because there were many cases of countries where data collection was omitted before March 11. For Australia, Chile, Denmark, France, Germany, Ireland, Italy, Luxembourg, Netherland, Spain and Sweden, we used values estimated by interpolation or extrapolation.

Subset Selection method. In our analysis of determinants, we next performed clustering analysis to find how the differences across countries due to determinants can be clustered. Specifically, we applied the DBSCAN method. DBSCAN refers to a density-based clustering method that performs clustering by measuring the density of an area. Other examples of such density-based methods are OPTICS, ADWICE and BIRCH (Ankerst et al., 1999; Buibeck and Nadim-Tehrani, 2004; Burbeck and Nadjm-Tehrani, 2007; Thang and Kim, 2011; Zhang et al., 1996) . Among these density-based methods, DBSCAN, a method developed by Ester et al. (1996) , is one of the most common clustering algorithms and is the most frequently cited method in scientific literature (Chakraborty and Nagwani, 2014) .

Through the above determinants analysis, we first reduced the range of variables to be considered in the analysis of the impact of the declaration, as shown in Fig. 2 . Furthermore, through cluster analysis, we narrowed the target of analysis to be used to test our hypotheses.

As shown in Fig. 2 , the analysis of the impacts of the declaration and RSV change is the second area of our research, aiming to test our established hypotheses. First, to test Hypothesis 1, we investigated the question "To what degree did the WHO pandemic declaration affect RSV?" According to the differences by country that appeared in the results of the first area of our analysis, we selected three countries to be representative of the clusters (United States, Germany and France) and analyzed them comparatively. We analyzed whether the change in the event week, more precisely, the change in RSV on March 12, was statistically different when viewed in a time series analysis; that is, we analyzed whether the effects of the intervention were observable. 3 It was possible that the RSV rise in the event week may simply be the extension of an existing trend and may not be a special increase in response to the pandemic declaration. Time series data often show rapid changes when affected by specific external events such as oil shocks, holidays, and policy changes. We refer to these external events as interventions, and the method of evaluating the effects of these external events is called intervention analysis (Box and Tiao, 1975; Montgomery and Weatherby, 1980; Ray et al., 2017) .

To test the second hypothesis, namely, to answer the question "Does an increase in RSV result in an increase in the detection of confirmed cases?" we performed the second analysis on the second subject, as described in Fig. 2 . We analyzed the correlation between RSV and new cases, statistically analyzing the relationship using the methods of crosscorrelation, Granger causality, and cointegration tests for analyzing VAR (Vector Autoregression) or VECM (Vector Error Correction Models) (Gharehgozli et al., 2020; Jun and Park, 2016; Jun et al., 2017) . In particular, we analyzed changes in COVID-19 testing following the increase in RSV to explain the correlation between RSV and new cases. The VAR model we used here is a multivariate time series model that combines the features of time series analysis and regression analysis to estimate the correlation and causal relationship between variables (Engle and Granger, 1987; Engle and Yoo, 1987) . In the cointegration test between variables, if there is a cointegration relationship between variables, VECM can be used in consideration of the long-term equilibrium. Therefore, instead of examining the relationship between variables with a VAR model, we only conducted cointegration analysis (Johansen, 1988; .

To evaluate these research hypotheses, we collected not only data from Google but also search statistics from the ECDPC (European Centre for Disease Prevention and Control) and the OECD, and data on new cases and new deaths and other information required for control from 37 OECD member countries. The time period subject to analysis was the period from February 1, immediately following the WHO's declaration 3 Although the WHO's announcement was made on March 11, we analyzed the RSV as of March 12. Because the standard time varies by country, and we also needed to consider the time of the release of WHO's announcement, we judged that March 12 would be the date that best reflects the public's response. Furthermore, we also considered that there were many countries in which the highest value of RSV (100) was reached on March 12, or in other words, in which the RSV showed a rapid increase, as shown in Table 2. that COVID-19's risk assessment at the global level was high, to May 11, which is two months from March 11, the date on which the WHO assessed COVID-19 to be a pandemic (WHO, 2020). Table 1 describes the variables included in this study, our sources and related references (OWID, 2020). 4 Here, we collected search information from 37 OECD countries via Google Trends, and since the data was collected by country, all search data was collected as daily relative search volume, that is, a normalized value based on the search volume on the day with the highest domestic search volume during the data collection period (with its value set as 100) (Jun et al., 2018) . Note that Google Trends provides RSV for "coronavirus," considered the thesaurus equivalent of COVID-19 (Google_Trends, 2020) . In the case of non-English speaking countries, statistics were also processed with the word "coronavirus."

The data on confirmed cases, deaths, and test recipients were also collected daily. As shown in Table 1 , such data was collected by the ECDPC from 37 OECD countries. However, collection of data was omitted on some dates, in some countries. For comparison of all these variables across countries, the data on both new cases and deaths were standardized to represent incidence per 1 million people. The number tested was standardized to be expressed as the rate of tests per 1000 individuals. 5 In the determinant factor analysis, due to the weak normality of the distribution, we converted the data to the square root values, as explained above. 6 Table 2 shows the changes in the RSV, the key variable of this study, from just before the event week to the mid-point, in the 37 OECD countries we analyzed. In addition, Table 2 shows the dates when the peaks of new cases and deaths existed during the analysis period. In Table 2 , we see that RSV showed rapid changes in many countries around March 11 and March 12. 17 out of 37 countries (45.9%) also exhibited their peak immediately following the declaration (within 2 days) and 28 out of 37 countries (75.7%) reached their peak of RSV within a week. These results confirm the likelihood that the WHO's pandemic declaration had a significant effect on the changes in RSV, while also indicating that there were differences among countries. Table 2 also presents the duration of the analyzed RSV. Duration is defined as the period (in days) until the RSV was restored to that of March 10, a date just preceding the event week. Table 2 also shows significant differences by country: it is notable that Italy, Japan, and Korea, where a large number of confirmed cases had already been reported, experienced a very short duration. The durations presented in Table 2 had a very high correlation with the RSV increase rate in the event week (Pearson correlation coefficient 0.808, p-value< 0.000), and this can be confirmed by referring to Fig. 3 . We thus confirmed that the greater the increase in RSV, the longer the duration, and in consideration of multicollinearity, we did not add RSV duration as a variable in our analysis.

Finally, to control the differences among countries, we considered several variables described in Table 1 . The variables included in this study were median age, the percentage of the elderly population (over 65 years old), GDP per capita, and hospital beds and the data were collected from sources including the UN, World Bank, and OECD. The definition of each variable is shown in Table 1 . GDP per capita refers to national income per capita, but the normality of the distribution was weak, so the value was converted to log values for our analysis. These variables were assumed to be constant values within each country during the period of 100 days analyzed, as shown in Table 2 .

As for the countries analyzed, as explained above, to analyze the first issue outlined in Fig. 2 we conducted a cross-sectional analysis of 37 OECD countries (including Colombia, since its participation was confirmed). To analyze the second issue explained in Fig. 2 , we needed to specify selected countries to account for national differences through longitudinal analysis, and for this purpose we chose the United States, Germany and France. The cluster of the highest interest was the sensitive cluster, and to represent this cluster we selected the United States since it visits, rather than the correlation between RSV and tests. 5 We chose to express the number of tests per one thousand persons, in comparison with the scale used for the figures for new cases and new deaths, in order to maintain the same scale as provided in the original data and also in consideration of the number of digits in the significant figures. 6 In this study, the data on deaths were relatively strongly affected by additional factors such as national testing capability, medical infrastructure, and age distribution of the population and there were also differences among countries during the targeted research period. Therefore, these values were used only for reference purposes. was one of the countries that responded most sensitively to the pandemic declaration of the WHO (showing the peak of RSV on March 12), and above all because detailed content analyses of Google searches were available for the United States (Cairo, 2020; Google_Trends, 2020) , allowing us to understand the changes in the context of searches. Germany was selected from the "insensitive" cluster and France from the "general" cluster; these selections took account of the ease of comparison with the United States based on factors such as economic size, population, and medical infrastructure. Fig. 4 compares the trends in RSV, new cases, and new deaths in the above three countries in the period of 100 days from February 1 to May 11, which is the target period for our analysis. The trends in the United States and Germany form a contrast, with the United States showing a remarkable increase in RSV in the event week, while Germany showed a relatively more sensitive reaction to other factors afterwards. The trends in new cases were also different: Germany's statistics fell sharply after peaking at the end of March, but the new cases in United States fell slowly. The trends in France was at an intermediate level compared to those in the United States and Germany. There was also a large difference in the trend of new deaths. In early April, France had the largest number, but the death toll fell dramatically soon thereafter. By contrast, the United States had a peak in deaths in late April, and thereafter the death count remained at a relatively steady level. Meanwhile, Germany had relatively fewer new deaths.

As explained in Fig. 2 , we analyzed significant determinants using the best subset selection method. According to the results presented in Table 3 , among the COVID-19 variables, the number of total tests (SQ) was found to be statistically significant in relation to the increase rate of RSV in the event week. Among the control variables, the proportion of the population over 65 years old was the only significant variable. Observing the direction of the coefficient, we found that as both variables decreased, RSV increased significantly. The model presented in Table 3 had an F-value of 14.315 (p-value <0.000), and therefore we judged this to be a statistically significant model. All VIFs (Variance Inflation Factors) were also less than 1.5, which was a strong result.

Furthermore, through the distribution of the studentized residual and P-P diagram analysis, we found that the presented model performed preprocessing well for outliers and leverage (James et al., 2013) , and therefore we concluded that there was no problem in using the determinants analysis results (model) indicated in Table 2 . The explanatory power (adj.R squared) of the model presented in Table 3 was 0.432, demonstrating a high level of explanatory power according to standards of social science research (Cohen, 1988) .

For reference, we applied the Lasso method to analyze the same independent variables. Based on these results, three variables (total tests (SQ), aged 65 older, and change of new cases) were judged to significant variables and the coefficient of each were respectively -0.018, -0.003, and 0.025. Compared to the results from best subset selection presented in Table 3 , the Lasso results were consistent with the significance and coefficient direction for the two variables that were shown to be significant in Table 3 . Although there were differences in the analysis results for the change of new cases, the Lasso method yielded the same results in the case of the second strong model, which we proposed based on the best subset selection method. Based on these findings, we judged that the results from the two methods did not differ widely. Also, as discussed above, we considered the fact that the Lasso method was inadequate for processing the missing values of the variables and that there was an insufficient number of training cases (37 countries) and consequently adopted the best subset selection results shown in Table 3 for our final analysis.

According to preceding studies, there are differences by country in the RSV responses to external events related to COVID-19 (Hu et al., 2020) . Therefore, there is a limit to generalizing from the hypothesis tests for specific countries. For this reason, to reduce and select the target of our hypothesis tests, we performed clustering analysis prior to analyzing the impacts of the declaration. To cluster the characteristics of countries, we examined the distribution of countries focusing on the total number of tests, which appeared to have a significant effect on changes in RSV, as shown in Fig. 5 . In contrast to Iceland, where there was a relatively high number of total tests and almost no change in RSV, most countries were clustered in the third quadrant, centering on Mexico, which is an example of a country where the total test number was relatively small but RSV rose significantly.

To more clearly visualize the differences between countries shown in Table 4 presents the result of clustering into four clusters according to the DBSCAN method. 7 Table 4 shows that the countries can be divided into 3 clusters excluding outliers. Based on total tests, which is also the most significant determinants variable in relation to our variable of interest (RSV change), the clusters that can be viewed as distinct were as shown in Table 4 . Cluster 0 represents the outliers, countries that are not included in any cluster. Most countries belonged to Cluster 1, consisting of countries in which the relationship between the rate of change in RSV and total tests was moderate, which we named the "general cluster." Cluster 2 is comprised of countries with a relatively high rate of change in RSV which we refer to as the "sensitive cluster," while Cluster 3 consists of countries with a low rate of change in RSV, which we call the "insensitive cluster." We selected the United States as the representative country for the sensitive cluster, Germany for the insensitive cluster, and France for the general cluster. Table 4 also provides a country-by-country ranking of the ratio of total tests (SQ) to the RSV change rate in the event week. Germany was ranked 6 th within the top group (1-12), France was 7th in the middle group (13-25), and the United States was 5th in the lower ranking group (26-37), and the fact that these three countries were each in the mid-range within each group was also considered in our country selection. 8 In the selection of representative countries, we also considered the population aged 65 and older, a control variable that showed a significant correlation in Table 3 . The variable percentile analysis results for the population aged 65 and older (Hinges of Tukey, excluding outliers) was 15.41% in percentile 25%, 19.68% in 75% and 21.36% in 90%. As seen in Table 2 , these values were almost the same as the "aged 65 and older" values of the United States (15.41%), France (19.72%), and Germany (21.45%), respectively.

We performed intervention analysis to statistically examine whether the RSV increase presented in Table 2 was caused by the pandemic declaration, that is, by an external intervention. First, we analyzed the United States, the representative country of the sensitive cluster. Intervention analysis requires the development of a time series model (ARMA) including interventions. Table 5 shows the prediction of the intervention model that would best predict the change in RSV in the United States during the 100 days of the target period of analysis (February 1 -May 11). 9 Insensitive 4 Germany (6), Norway (5), Slovenia (7), Switzerland (4) Note: () is the ranking of countries based on the ratio of total tests (SQ) to the weekly change rate of RSV 8 We did not perform clustering based on the ratio of total tests (SQ) to the rate of change in RSV in the event week because it is difficult to distinguish when the extreme value, the denominator and the numerator are simultaneously small (Japan), as shown in Table 5 . We used the DBSCAN method so that we would be able to consider the size as well as the ratio. 9 In Table 5 , the variable X which indicates the intervention effect had the value of 1 only on March 12; on other days, it had a value of 0. SIGMASQ is the estimate of the variance of the innovations and Table 5 shows the results calculated by Eviews10.

According to the data on COVID-19 searches in the United States shown in Table 2 , the RSV was 100 on March 12, immediately following the declaration on March 11. Compared to the RSV on March 10 and March 11, which were respectively 45 and 57, this represents a very dramatic increase. The effects of these interventions are also confirmed statistically; as shown in Table 5 , the time series models explaining the RSV in the United States are the AR(1), AR(2), and MA(3) models and here too, the intervention effect (X) on March 12 was found to be statistically significant. 10 According to the intervention model presented in Table 5 , the effect of the intervention (coefficient) may be judged to have had the effect of further increasing the RSV by 20.3 on top of the existing trend of increase. This confirms it had the effect of additionally increasing searches by 25.3% on the single day of March 12.

The left side of Fig. 6 shows the intervention model predicted as static and the measured values. It can be seen that the intervention model's predictive power is strong. On the right, we compare the measured values with the dynamic model in which we assume the effects of intervention did not exist in the intervention model. The RSV of March 12 predicted by the intervention model was close to 80 (left), while the RSV is close to 60 in the non-intervention dynamic intervention model (right), demonstrating again that there is a difference of 20 in the RSV. 11 If we observe the cumulative difference between the measured RSV values and the RSV values predicted assuming nonintervention, for the period of 2 months from March 12 to May 11, there is a difference of 21.9%, which again confirms that the effect of the intervention (declaration and follow-up measures) on increasing RSV is similar to the level shown in Table 5 . Table 2 shows that RSV searches in Germany, the representative country of the insensitive cluster, rose after the declaration on March 11 and peaked on March 13, but unlike in the United States, the RSV did not rise to 100. Germany's RSV peak emerged a while after the event week, on March 22, when the government and the federal states announced a policy strongly prohibiting gatherings (Ohms, 2020) . The weak effect of this intervention can also be verified statistically. As shown in Table 6 , the time series model that explains the German RSV is the AR(1) model, where the intervention effect of March 12 is not statistically significant and there was hardly any effect (coefficient). 12 Thus the intervention effect of the WHO pandemic declaration observed in the United States was not observed in Germany, which belongs to the insensitive cluster. Rather, the intervention effect of another external event (on February 26), the emergence of new patients within two weeks (Ohms, 2020) , was statistically significant with a significance level of 10%, forming a clear contrast.

Finally, Table 7 presents the results of developing the intervention model for France, which belongs to the general cluster. The RSV trends in France shown in Table 2 indicate that the RSV was 44 on March 10, similar to that of the United States, and increased immediately after the declaration on March 11, but the rate was slower than in the United States; the RSV in France reached 100 only on March 15. According to the results of the development of the RSV prediction model for France presented in Table 7 , AR(1) and AR(4) are significant models, and here the intervention effect of March 12 was also statistically significant. 13 As indicated by the intervention model presented in Table 7 , the effect of intervention (coefficient) additionally increased the RSV by 9.5 above the increase in accordance with the existing trend, and as of March 12, there was an additional rise of 12.6% in the RSV. Although this rise is lower than that of the United States, it confirmed the statistically significant effects of intervention.

In the same manner as Fig. 6, Fig. 7 compares the prediction results of the intervention model and the non-intervention model. When looking at the cumulative difference between the measured RSV value and the RSV predicted based on non-intervention over the two months from March 12, there was a difference of 20.5%, which was similar to that of the United States.

Synthesizing these findings, we can see that the effect of intervention was statistically significant in the United States (sensitive cluster) and France (general cluster), allowing us to accept the first hypothesis, namely that the external intervention by the WHO declaration will strengthen public awareness.

4.2.2.1. RSV and new tests. As discussed above, the WHO pandemic declaration had a significant effect on raising RSV, that is, on strengthening public awareness. As reported in preceding studies, before and after the WHO pandemic declaration, statistics of Google search queries showed that in addition to queries asking "What is a pandemic?" queries regarding "How to get tested for coronavirus?" also increased rapidly (Cairo, 2020) . This indicates that the WHO pandemic declaration not only had the effect of enhancing public awareness, leading to simple information searches, but also may have led to actions, as explained in the research model presented in Fig. 1.   Fig. 6 . Comparison of intervention model predictions and measured values (left) and comparison of non-intervention model predictions (right) -United States. 10 In the residual analysis, because the p-values of the Q statistic were all insignificant at 5%, the null hypothesis that these white noise terms are statistically independent was adopted (failed to reject). 11 Non-intervention model refers to a model in which the effect of intervention is eliminated from the developed intervention model (that is, all x values are zero). 12 Changing the intervention date to March 11 or 13 did not have any statistical significance. The residual analysis in Table 6 also adopted (failed to reject) the null hypothesis that "white noise terms are statistically independent." 13 In the residual analysis, because the p-values of the Q statistic are all insignificant at 5%, the null hypothesis that these white noise terms are statistically independent was adopted (failed to be rejected). Fig. 8 presents comparatively how this interest in testing is reflected in the RSV. In the United States and Germany, searches for COVID-19 overall and specifically for COVID-19 tests showed similar trends, and there was a strong tendency to search for test information more intensively before and after the WHO pandemic declaration (note the sharper peaks). Table 8 shows that there is almost no time lag statistically, and in cross correlation, there is no time lag present and the Granger causality also existed with a very small time lag. 14 This confirms that in the United States and Germany the interest in testing increased almost at the same time, in the same pattern, as the increase in RSV. However, in the case of France, there was a difference in the pattern, as the interest in testing began to appear somewhat later. In the case of France, cross correlation showed the highest correlation with a 6-day lag, and the Granger causality also had the most significant relationship with the same 6-day lag (F-statistic: 6.982, p-value: <0.000). These results demonstrate that the WHO pandemic declaration caused an increase in COVID-19-related RSV, and while there may be some time lags, specific awareness about COVID-19 testing was indeed included in the pattern.

To determine whether such interest in tests was linked to the action of taking the COVID-19 test, we examined statistics on new tests as shown in Fig. 9 . In Fig. 9 , the values in the first period indicate the total number of tests up to just before the event week, and the values presented thereafter are the status of weekly new tests. In all three countries, in the event week, more tests were conducted in that one week than the total number of tests preceding. According to the duration values presented in Table 2 , which is related to the correlation between RSV and the number of new tests, the period of the steepest increase was one week in Germany, the country which had the shortest duration of 9 days as seen in Table 2 . By contrast, France, with a duration of 16 days, showed a two-week spike. The United States, which had the longest duration of 23 days, continued to show a surge for three weeks. This confirms that actual test taking increased rapidly after the rise in interest or perception regarding testing. The form was similar to the continuous response form seen in pulse intervention, where responses manifest continually after the temporary intervention. 15 Based on the above results, we concluded that the increased interest in the tests seen in Fig. 8 was associated with the rapid increase in new tests shown in Fig. 9 .

Observing the relationship between RSV and new cases in the United States shown in Fig. 4 , we can see that the change in RSV tends to precede the latter, resulting in some time lag. According to the cross-correlation presented in Table 9 , RSV precedes the latter and has a high correlation in the positive direction with a lag of 23 days. 16 The presence of a positive correlation indicates that this correlation may be the effect of rapid testing due to increased awareness. As for Granger causality, Granger causality was observed at the significance level of 0.05 at the 12-day lag, while at the 21-day lag, and a significant Granger causality was observed at the 10% significance level.

We confirmed that the first difference is needed in the unit root test and took the difference and analyzed the VAR model (proceeding to 24 15 During the analysis period in Germany and France, test data were obtained only on a weekly basis, and therefore we did not perform any statistical reviews. In the case of the United States, which was the only country for which daily data was available, the correlation between RSV and new tests was not significant in the VAR model. This is because daily new tests tended to increase almost linearly during the analysis period. 16 In Table 9 , for cross correlation, only the lags with the highest correlation are displayed. For Granger causality, from among the analysis results of up to 24 lags, only the values for the interval with the lowest p-value and values for additionally significant intervals are presented.

lag intervals considering the Granger causality, etc.). The results confirmed a strong autocorrelation between RSV and new cases and showed that RSV had a significant correlation only after 24 days (coefficient: 0.568, t-statistics: 2.036). As described above, when a significant relationship appears in cointegration, the VECM analysis is superior to the VAR analysis for identifying the long-term equilibrium relationship. Table 10 shows the result of the Johansen cointegration test performed using the VECM model: we found one or more cointegration equations, confirming that new cases and RSV have a long-run relationship. Taken together, these findings indicate that in the United States, RSV had a statistically significant correlation with new cases, and the significant correlation was present with an approximately 3-week lag.

In the case of Germany, as shown in Fig. 4 , RSV tended to precede new cases by several weeks. According to the cross-correlation in Table 9 , RSV precedes and there is a high correlation in the positive direction with a lag of 19 days. As for Granger causality, we see that there is a very strong Granger causality (significance level of 0.000 or less) at a lag of 4-6 days and a significant Granger causality existed at the 0.05 significance level up to a maximum of a 15-day lag. According to the Johansen cointegration test results presented in Table 10 , there is one cointegration equation, and there is indeed a long-run relationship between new cases and RSV. In sum, in Germany, RSV had a statistically significant correlation with new cases, with the significant relationship showing after a lag of about 1 to 2 weeks.

In the case of France as well, RSV preceded new cases by several weeks as shown in Fig. 4 . According to the cross-correlation in Table 9 , RSV precedes and the correlation was highest in the positive direction after a lag of 17 days. As for the Granger causality, a significant Granger causality was observed at the significance level of 0.05 even at the 6-day lag but very strong Granger causality (significance level less than 0.000) was observed multiple times at a lag of around 20 days. According to the Johansen cointegration test results presented in Table 10 , there is one cointegration equation, and there is a long-run relationship between new cases and RSV. In conclusion, in France, RSV also had a statistically significant relationship with new cases, with the significant correlation emerging with a lag of about 3 weeks.

Synthesizing the above results, we can conclude that although the degree of lag may have varied, in the United States (sensitive cluster), France (general cluster), and even Germany (insensitive cluster), RSV was able to explain the change in new cases, and therefore the second hypothesis, that "increased public awareness will increase the identification of confirmed cases," can be accepted. 17 Table 11 presents the test results for the hypotheses we set up in Section 3.

In the cross-sectional study, the variables that were found to strongly influence the rate of change in RSV immediately after the WHO pandemic declaration were total tests and the proportion of the population over 65 based on the best subset model, as shown in Table 3 . The direction of influence all appeared to be negative. This can be attributed to the fact that in countries where there were already a high of total tests, the immediate response to the WHO pandemic declaration was weaker because the public awareness was already relatively high. The 17 Since the case of Germany was rejected in Hypothesis 1, the discovery of new cases due to the increase in RSV should be interpreted as a result of the existing awareness-raising efforts, reflecting the existing trend, rather than as an outcome of the declaration. relatively low rate of increase in RSV when the proportion of the elderly population was high may be attributed to differences in information search activity by age (Perrin, 2015) and may also be explained by the high proportion of the elderly in confirmed cases (Kass et al., 2020) .

Therefore, the response of the WHO pandemic declaration is bound to be different from country to country due to differences in total tests and demographic distribution of the population by age group. As shown in Fig. 5 , when the total tests are high, the increase rate of RSV is relatively low even in the wake of the pandemic declaration. This difference can be attributed to the fact that where awareness had already been high, and there is no strong surge in interest or intense information searching, as explained in Fig. 1. In Fig. 9 , we saw that Germany, which had a relatively high number of total tests, had a smaller increase in RSV compared to the United States or France, and that Germany belonged to the insensitive cluster.

As shown in Table 5 , our analysis of the United States, in the cluster of interest, demonstrated that the WHO intervention had a statistically significant effect on the increase in RSV. We found that the RSV immediately increased by 20.3%, heightening public awareness (or attention), and the difference in RSV during the analysis period following the declaration (up to May 11) also indicates that public awareness increased by 21.9%. By contrast, in Germany, which belongs to the insensitive cluster, the declaration of the WHO showed no statistically significant effect (refer to Table 6 ). In France (general cluster), the WHO intervention had a statistically significant effect on the increase of RSV (refer to Table 7) , and raised the public awareness which was at 9.5%, a relatively low level; during the analysis period, public awareness also increased by 20.5%. Thus we may conclude that in the cluster of interest and the general cluster, the WHO pandemic declaration increased public awareness by an additional 10-20%, had a positive effect leading to an increase in tests, which in turn enabled the quick identification of new cases.

Regarding new tests, although in Table 8 we were able to statistically explain the increased interest in tests, in Fig. 9 , the statistical relationship with RSV was unclear. This phenomenon can be explained by the lack of preparation in terms of test infrastructure. The heightened interest in testing shown in Fig. 8 could not be directly converted into actual testing due to the lack of test infrastructure. As explained in Fig. 9 , the increased interest in testing instead led to a slower rise in new tests. Even when searches are made for information regarding COVID-19 or testing, actual access to COVID-19 tests may have varied depending on the country. Fig. 10 presents results which indicates these differences in test accessibility by country, comparing the percentage of positive test results (cases) by country. As shown in Fig. 10 , all three countries exhibited a continual rise in the positive rate until the third week after the event week, which confirms the likelihood that all three countries lacked the infrastructure to administer sufficient testing for symptomatic individuals up to the beginning of April. Fig. 10 can be effectively used to explain the relationship between RSV and new cases, as described in Table 9 . As explained in Tables 9 and 10, RSV was found to be significant in explaining new cases in all three countries. Furthermore, we explained that there was a lag of about 3 weeks in the United States and France and a lag of about 1 to 2 weeks in Germany, and these lags shown in Fig. 10 were affected by the shortage of test infrastructure. In Germany, where there was a relatively high number of total tests and where the positive case rate was low, the lag between the trends of RSV and new cases was short.

We can draw several policy implications from these research results. First, while the WHO pandemic declaration also contributed to initiating emergency measures in each country, it also played a critically important role in raising public awareness. In other words, it contributed to public awareness which induced symptomatic patients to take tests, helping to identify confirmed cases early and treat and isolate patients. Therefore, when considering the timing of its pandemic declaration, the WHO should consider not only the trends in new cases and the number of countries with cases (Mackenzie, 2020; Piper, 2020) , but also consider its impact on public awareness. In seeking to raise public awareness, the WHO should minimize countries where intervention would be meaningless and maximize the effectiveness of intervention, Note: * denotes rejection of the hypothesis at a significance level of 0.05.

Hypothesis contents and test results.

No.

Test results

The intervention of the WHO in the form of the pandemic declaration will increase public awareness.

Cairo (2020) 

Partially accepted (The hypothesis was accepted in the cases of the United States and France, representing the sensitive cluster and the general cluster; it was rejected in the case of the insensitive cluster, namely Germany) 2

Increased public awareness will result in an increase in the discovery of confirmed cases.

Jun et al., by building strategic alliances with each country (e.g., through test infra readiness), and for this purpose, real-time changes in RSV may be useful. Secondly, this study confirmed that the WHO's announcement did not have a similar impact on all OECD countries in terms of public awareness, and that the impact was bound to vary depending on the number of tests conducted before the announcement or the number of confirmed cases. Since the WHO cannot make a pandemic declaration at the optimal time for all countries, the efforts of individual countries are also important. Compared to the countries where the increase in RSV was relatively high, those countries where the RSV surge lasted shorter will need to make efforts to provide information even more promptly, because the duration in which awareness precipitates information searches was shorter in such countries. As shown in Table 2 or Fig. 3 , if the rate of increase in RSV after the declaration is relatively low, the duration will be short, and therefore public interest will last only a short time. Taking this into account, it would be possible to efficiently control the speed of policy actions and information dissemination.

Thirdly, if the immediate response and duration of information searches indicate the level of awareness that existed in each country up to the immediately preceding time point, RSV can henceforth also be used for ex post evaluation of each country's COVID-19 response efforts. For example, Fig. 5 shows that Denmark, Israel, and Japan had belowaverage numbers of total tests and as seen in Table 2 , these countries had a lower RSV increase rate and shorter duration: this may indicate that these countries had already been more successful in raising public awareness.

Finally, because elderly citizens may not engage in searches as actively as younger people to improve their awareness, a separate public awareness effort may be needed to reach this demographic group. Table 3 indicated that when the proportion of the elderly population was high, the information search response was low. This may be a countryspecific difference or a difference in the information search channels used by specific age groups (Perrin, 2015) , but since it may also be a global characteristic of the elderly age group or an indicator of their lack of public awareness, it is necessary to consider policy initiatives to raise public awareness among the elderly and provide appropriate information.

This study demonstrated that the WHO pandemic declaration increased public awareness and increased searches for information on COVID-19, contributing to the rapid rise of RSV. We also found that this surge in RSV enhanced understanding of COVID-19 and raised interest in testing which in turn may have made a significant contribution to identifying new cases through the COVID-19 tests.

The scholarly significance of this study is that it proposed a method of quantitatively observing the effects of specific external events, more specifically, global policy reactions, on the general public by using the social big data of RSV. It is also significant that this study objectively illuminated the series of processes beginning from awareness and extending to information search and public quarantine activities (testing). In terms of policy and practical application, our findings significantly demonstrated that the WHO's involvement had a direct impact on the public as well as on policy-making authorities, suggesting the need for reinterpretation of the role of WHO. Another contribution of this study is that it suggests the potential benefits of governments utilizing RSV when implementing policy in response to the outbreak of infectious diseases such as COVID-19.

One limitation of applying the results of this study is that the RSV responses to the WHO pandemic declaration may not only reflect rational judgments, as explained in general human behavior analysis. Risk awareness inevitably includes the component of emotional reactions, and an abnormal rise in information search, referred to as hype (Fenn and Raskino, 2008; Jun, 2012b; Katafuchi et al., 2020) , may be reflected in the results. There had already been a demand from the international community for the pandemic announcement since January 2020 (Piper, 2020) , which means that the WHO pandemic declaration may have incited fear in a situation where public risk awareness had already been heightened. This hype phenomenon does not significantly change the conclusions of this study, but it will be necessary to develop a new predictive model when making forecasts regarding the period following the subsidence of the emotional disturbance.

This research was supported by Korea Institute of Science and Technology Information (KISTI).

The raw data of the COVID-19 used in this study was received by OWID (https://github.com/owid/covid-19-data).

Authors Term

Jae-Seong Lee

None.

Prediction of dengue incidence using search query surveillance

Google searches accurately forecast RSV hospitalizations

OPTICS: ordering points to identify the clustering structure

Predicting COVID-19 incidence through analysis of google trends data in iran: data mining and deep learning pilot study. JMIR Public Health Surveillance 6, e18828. BBC_news, 2020a. Coronavirus: Germany to Impose Border Controls Over Coronavirus

Spain and France Announce Sweeping Restrictions

Assessing global COPD awareness with Google Trends

Intervention analysis with applications to economic and environmental problems

A critical appraisal of the world health organization's international health regulations (2005) in times of pandemic: it is time for revision

Covid-19

ADWICE-anomaly detection with fast incremental clustering

Adaptive real-time anomaly detection with incremental clustering

When Google got flu wrong

Searching for Covid-19. Schema Design, Google News Initiative Axios

Mobilizing policy (in) capacity to fight COVID-19: understanding variations in state responses

Analysis and study of Incremental DBSCAN clustering algorithm

Real-time forecasts and risk assessment of novel coronavirus (COVID-19) cases: A data-driven analysis

Social distance and SARS memory: impact on the public awareness of

Statistical Power Analysis for the Behavioral Sciences

Assessing google flu trends performance in the United States during the 2009 influenza virus A (H1N1) pandemic

Coronavirus: France imposes 15-day lockdown and mobilises 100,000 police to enforce restrictions

Coronavirus: France to Close All Schools and Universities, Macron Says

Using internet search behavior to assess public awareness of protected wetlands

Google Flu Trends: correlation with emergency department influenza rates and crowding metrics

Co-integration and error correction: representation, estimation, and testing

Forecasting and testing in co-integrated systems

A density-based algorithm for discovering clusters in large spatial databases with noise

Mastering the Hype Cycle: How to Choose the Right Innovation at the Right Time

The spread of awareness and its impact on epidemic outbreaks

Impact of COVID-19 on the economic output of the US outbreak's epicenter

Detecting influenza epidemics using search engine query data

Timeline of the coronavirus pandemic and U.S. response. Just Secur

Coronavirus is a Huge Topic in Search Interest

Two-stage effects of awareness cascade on epidemic spreading in multiplex networks

How people emotionally respond to the news on COVID-19: an online survey

Extended comparisons of best subset selection, forward stepwise selection, and the lasso

More effective strategies are required to strengthen public awareness of COVID-19: evidence from google trends

Applications of google search trends for risk communication in infectious disease management: a case study of COVID-19 outbreak in Taiwan

An Introduction to Statistical Learning

Statistical analysis of cointegration vectors

A comparative study of hype cycles among actors within the sociotechnical system: with a focus on the case study of hybrid cars

An empirical study of users' hype cycle based on search traffic: the case study on hybrid cars

Consumer information search behavior and purchasing decisions: empirical evidence from Korea

Forecasting by analogy using the web search traffic

A study of the method using search traffic to analyze new technology adoption

Ten years of research change using Google trends: from the perspective of big data utilizations and applications

A study on the effects of the CAFE standard on consumers

Reappraising the utility of google flu trends

Obesity could shift severe COVID-19 disease to younger ages

COVID-19 with stigma: theory and evidence from mobility data

Marketing management 14/e

Advances in nowcasting influenza-like illness rates using search query logs

The parable of Google Flu: traps in big data analysis

Retrospective analysis of the possibility of predicting the COVID-19 outbreak from Internet searches and social media data

The impact of COVID-19 epidemic declaration on psychological consequences: a study on active Weibo users

Trump declares national emergency -and denies responsibility for coronavirus testing failures

Covid-19: why won't the WHO officially declare a coronavirus pandemic? Newsscientist

Modeling and forecasting time series using transfer function and intervention methods

Airport risk of importation and exportation of the COVID-19 pandemic

The use of google trends in health care research: a systematic review

Kontaktverbote zu Mehr als Zwei Personen, Friseure zu -Darauf Haben Sich Bund und Länder Geeinigt

Reassessing Google Flu Trends data for detection of seasonal and pandemic influenza: a comparative epidemiological study at three geographic scales

More diseases tracked by using google trends

Social media usage

Why the WHO waited to call the coronavirus a pandemic

Evaluating Google Flu Trends in Latin America: important lessons for the next phase of digital disease detection

Adaptive nowcasting of influenza outbreaks using Google searches

Herd Immunity: Understanding COVID-19

Technology forecasting using time series intervention based trend impact analysis for wheat yield scenario in India

Diffusion of Innovations

What can digital disease detection learn from (an external revision to) Google Flu Trends?

The utility of "Google Trends" for epidemiological research: lyme disease as an example

Search Engine Market Share Worldwide

Investigating the utility of Google trends for Zika and Chikungunya surveillance in Venezuela

The anomaly detection by using dbscan clustering with multiple parameters

UPDATE: All German states announce school and Kita closures

2020. I cannot understand reasons for the exception of Great Britain

The use of google trends to investigate the loss of smell related searches during COVID-19 outbreak. International Forum of Allergy & Rhinology

High temperature and high humidity reduce the transmission of COVID-19

The impact of awareness diffusion on SIR-like epidemics in multiplex networks

Q&A: What you need to know about the United Kingdom and Ireland travel ban

The impact of awareness on epidemic spreading in networks

Effects of temperature and humidity on the daily new cases and new deaths of COVID-19 in 166 countries

Accurate estimation of influenza epidemics using Google search data via ARGO

The effects of global awareness on the spreading of epidemics in multiplex networks

BIRCH: an efficient data clustering method for very large databases

He received his Ph.D. from the Science and Technology Studies Program at Korea University. His area of interest includes the demand forecasting, emerging technology detecting and new technology adoption analysis

Hyoung Sun Yoo is a principal researcher at the Korea Institute of Science and Technology Information (KISTI). Also, he is an assistant professor of Science and Technology Management Policy in University of Science & Technology (UST). His area of interest includes demand forecasting, science and technology policy, complex network, and agent-based modeling

His areas of interest include National Innovation Systems (NIS), R&D project management, R&D collaboration management, innovation policies for SMEs, and Decision-making Support Systems (DSS) based on machine learning and Artificial Intelligence

Seung-Pyo Jun, the first and corresponding author, acknowledges support from the Creative Challenge Research Program at Korea Institute of Science and Technology Information. The authors would like to express special gratitude to OWID (Our World in Data team) for sharing valuable COVID-19 data for our research and to our reviewers and editors who have generously provided helpful feedback to improve this paper.