key: cord-0891564-id8os59s authors: Kłak, Anna; Grygielska, Jolanta; Mańczak, Małgorzata; Ejchman-Pac, Ewelina; Owoc, Jakub; Religioni, Urszula; Olszewski, Robert title: Online Information of COVID-19: Visibility and Characterization of Highest Positioned Websites by Google between March and April 2020—A Cross-Country Analysis date: 2022-01-28 journal: Int J Environ Res Public Health DOI: 10.3390/ijerph19031491 sha: f5578e2b47771718bc5e28eb0a1487dfc73e008b doc_id: 891564 cord_uid: id8os59s Background: The WHO has used the term “infodemic” to describe the vast amount of false and true information that was making it difficult for people to find reliable information when they needed it. The infodemic spreads faster than COVID-19 itself. The main objective of the study was to characterize and analyze content about COVID-19 returned by Google during the pandemic and compare it between countries. Methods: The study was conducted between 30 March and 27 April 2020. The information was searched through local Google websites using the “COVID-19”, “Coronavirus”, “SARS-CoV-2” and “fake news” keywords. The search was conducted in Australia, France, Germany, Italy, Poland, Singapore, Spain, UK and the USA. The total number of the analyzed webpages was 685. Results: The most frequent types were News websites 47% (324/685) and Governmental 19% (131/685) while the least were Health portals 2% (17/685) and Scientific journals 5% (35/635), p < 0.001. United States and Australia had the highest share of Governmental websites. There was a positive correlation between the amount of preventive information and a number of SARS-CoV-2 infections in countries. The higher the number of tests performed, the higher was the amount of information about prevention available online. Conclusions: Online information is usually available on news and government websites and refers to prevention. There were differences between countries in types of information available online. The highest positioned (the first 20) websites for COVID-19, Coronavirus and SARS-CoV-2 keywords returned by Google include true information. Globalization, common access to the Internet and the rapid development of social media make fake information easy to spread. The phenomenon is so widespread that the English phrase "fake news" has made it its way to other languages. "Fake news" is defined as: "fabricated information that imitates news media content in form but not in organizational process or intent, which overlaps with other information disorders, such as misinformation (false or misleading information) and disinformation (false information 3 of 26 was 5-6 times. In Italy, Spain, France and Germany, the increase was more than double. The largest differences were observed in Singapore (more than a 14-fold increase), but a significant increase in the incidence did not occur until 20 April. The increase in infections was the slowest in Australia (133%); usually it was a double-digit daily increase, with the highest recorded amounting to 266 new cases (2.04). The number of deaths varied from country to country. The highest percentage of deaths as of 1 April 2020 in relation to cases was recorded in Italy (11.9%), Spain (9.0%), Great Britain (8.7%), France (7.1%). These values were higher than the world average (5.2%). In other countries, this ratio was lower than the world average and amounted to: USA: 2.9%, Poland: 1.8%, Australia: 0.5% and Singapore: 0.3% [11, 13] . The aim of the study was to characterize and analyze the pages of the first 30 search engine results (SERP) for each country publicly available during the pandemic, and to compare them between countries. Detailed objectives included: -Characterization and analysis of websites typology by Google based on "COVID-19", "Coronavirus", "SARS-CoV-2" and "fake news" key words; -Characterization and analysis of online information returned by Google based on "COVID-19", "Coronavirus" and "SARS-CoV-2" key words; -Calculating frequency of fake news on "COVID-19", "Coronavirus", "SARS-CoV-2" key words; -Characterization and analysis of online information returned by Google based on "fake news" key words; -Analysis of associations between results and epidemiological data on COVID-19, such as: number of deaths, number of infections, number of SARS-CoV-2 tests performed; -Characterization and analysis of online information returned by Google based on "COVID-19", "Coronavirus", "SARS-CoV-2" and "fake news" about celebrities, religion and testimonials; -Analysis of the Journal of the American Medical Association (JAMA score). The research question was whether there were differences between the countries in content available online about COVID-19. The study was conducted in 2020 between March 30 and April 27 using the methodology employed by Arif N. et al. [7] for analyzing content about immunization. Authors used "vaccines" and "autism" keywords in Google between June and September 2017 and categorized the websites. The typology included: Commercial (C), Government (G), Health portal (HP), News (N), Non-profit (NP), Professional (P), Scientific journals (SJ) and "others" (O).The JAMA score was used (for the presence of the following information: author, date, references, owner of website) and the webpages were annotated according to the following features: (1) the name of the vaccine mentioned; (2) the overall stance on vaccines (positive, negative, or neutral); (3) the chemicals or adjuvants mentioned; (4) whether the page mentioned complementary and alternative medicine (CAM); (5) whether religion was mentioned; (6) whether the page contained a testimonial; (7) whether a celebrity was mentioned [7] . This study was carried out according to SRQR guidelines (standards for reporting qualitative research) [27] . The research was carried out using both Google Chrome and Firefox browsers without any add-ons. All cookies and the browser history were deleted prior to the search process to avoid any bias in the results [7] . The researchers were aware that Google may identify the user's location using their IP address and so influence the results, so they did not use a VPN while searching. The following searches were conducted: Poland (google.pl), France (google.fr), Italy (google.it), USA (google.com), UK (google.co.uk), Singapore (google.com.sg), Germany (google.de), Spain (google.es), Australia (google.com.au). All SERPs for each country and one keyword were analyzed on the same day. This part of the methodology is important in order to calculate the return frequency of websites in relation to their type. If the content returned was too extensive to analyze in one day (one keyword for a country), the links to the first 30 SERPs were copied to Excel and analyzed over two days. As the search process was performed in Poland, the research team deleted the browser history and all cookies, and reset Google to the selected country and language. Although the keywords were typed in English, Google was set to return results in the national languages. The country Google search engines were used and four key words: "COVID-19", "Coronavirus", "SARS-CoV-2" and "fake news". All but one national language in the study use the same spelling for "coronavirus". The local word "koronawirus" was used for Poland. In Singapore, which has four official languages including English, the returned results using "coronavirus" were considered equal and comparable to other countries. It was decided to use only those keywords as we planned to obtain a sample of the websites returned independently for each expression. Thus, we did not to use questions such as "mitigations measures?", "auto tests?", "how to treat COVID-19" or "how to protect against coronavirus" because the results would be different depending on an exact question. We decided to use the search terms "COVID-19", "Coronavirus", "SARS-CoV-2" and "fake news" as this best represents what the lay public would search on the Internet. The researchers had basic and advanced language skills in the languages of the analyzed countries. Unclear content was translated into English or was consulted upon with translators. The researchers deliberately used the keyword "fake news" because at the time the study was conducted, it was popular on the web to comments on fake news about the COVID-19 pandemic. However, a detailed analysis showed that some of the fake news about the COVID-19 pandemic was not debunked, but accepted as real information. Three researchers analyzed the first 30 SERPs for three different countries at the same time. There was no more information on the Internet about the topic at the time the study was carried out. By design, 30 SERPs were taken into account after the initial research. After compiling items for different countries in their national languages (in Singapore-English), it turned out that getting 30 addresses for each keyword is impossible. In the end, the first 20 SERPs were taken into account, which for some countries and for this reduced number was impossible to obtain (fake news: Spain-16 pages, Poland-13, see: Table 1) . Each researcher had to analyze 4 key words for each country, i.e., 120 SERPs and 360 SERPs in total. Each researcher analyzed one keyword a day for a country. Whenever the content was too extensive, the links to SERPs were copied to Excel and investigated the next day. The overall number of the analyzed SERPs reached 685 with the following numbers for each of the countries: Poland: 73, France: 80, Italy: 80, UK: 61, Singapore: 81, Germany: 80, Spain: 76, Australia: 80, USA: 74. Using a 0-1 system, the researchers assessed whether each type of information appeared on the SERP, where 0 means no information, and 1 means it was present. Moreover, each SERP was rated according to webpage typology. The fact that the information on the website was "fake news" was also assessed using 0-1, where 0 means real information and 1 means fake news. The researchers, by consensus, judged whether the information was fake news or not, on the basis of information posted on websites that were generally considered reliable. Such websites include: the World Health Organization, WHO; the Centers for Disease Control and Prevention, CDC; the National Health Service, NHS; or national health authorities. Additionally, it was necessary for the team to agree on the comparability of the published information in different countries on the same topic. A structured study protocol (flowchart) is presented in Table 1 and the Supplementary Materials (Tables S1 and S2 ). 0 0 14 3 0 3 0 20 total 9 1 30 7 2 20 12 81 USA COVID-19 11 0 2 2 1 4 0 20 Coronavirus 13 0 1 1 4 1 0 20 SARS-CoV-2 2 1 0 0 2 4 11 20 Fake news 0 0 9 1 0 4 0 14 total 26 1 12 4 7 13 11 74 Australia COVID-19 15 0 4 0 1 0 0 20 Coronavirus 12 0 5 1 1 0 1 20 SARS-CoV-2 4 0 5 2 5 3 1 20 Fake news 0 0 18 0 2 0 0 20 total 31 0 32 3 9 3 2 80 Poland COVID-19 4 3 10 0 0 3 0 20 Coronavirus 2 2 13 0 1 2 0 20 SARS-CoV-2 4 3 9 0 3 1 0 20 Fake news 0 0 5 1 1 6 0 13 total 10 8 37 1 5 12 0 73 UK COVID-19 2 0 2 7 4 4 0 19 Coronavirus 2 0 14 0 2 1 1 20 SARS-CoV-2 1 2 0 0 1 3 5 12 Fake news 0 0 8 1 0 1 0 10 total 5 2 24 8 7 9 6 61 Italy COVID-19 6 0 5 0 5 3 0 19 Coronavirus 6 0 12 0 1 1 0 20 SARS-CoV-2 2 0 11 0 7 0 1 21 Fake news 1 0 18 1 0 0 0 20 total 15 0 46 1 13 4 1 80 France COVID-19 4 0 14 0 1 1 0 20 Coronavirus 3 0 16 0 1 0 0 20 SARS-CoV-2 4 0 9 2 4 0 1 20 Fake news 0 0 20 0 0 0 0 20 total 11 0 59 2 6 1 1 80 TOTAL N 131 17 324 31 75 73 34 685 TOTAL % 19% 2% 47% 5% 11% 11% 5% 100% Values indicate the number of webpages in each SERP. Researchers have jointly defined the scope of the analysis based on the information on COVID-19 available in the public domain of the Internet methods used by Arif N. et al. [7] . Each website was assessed according to the information it contained. The following content was analyzed: information on quarantines, disease/infection symptoms, disease/infection risk factors, disease/infection consequences; virus transmission routes; virus incubation period; virus carrier; disease treatment; prevention measures prior to infection; alternative/supplementary medicine-unconventional or "home" treatment (including attitudes towards alternative medicine: positive, negative, neutral); epidemiological data (number of cases, deaths etc.); whether the page contained a testimonial (e.g., a personal story); whether a celebrity was mentioned; whether a religion was mentioned; others such as: regulations, services, economy, information on online fake news. The "other" category also included information on disease etiology. The etiology was not initially considered a separate category; however, numerous websites included speculation on how the pandemic started. All such speculation was eventually classified as "other". All the websites returned by Google using the "COVID-19", "Coronavirus" and "SARS-CoV-2" keywords were judged as true or false according to the scientific knowledge provided by organizations such as the CDC; NHS; Chief Sanitary Inspectorate "Główny Inspektorat Sanitarny, GIS"; National Health Fund "Narodowy Fundusz Zdrowia, NFZ"; Ministry of Health "Ministerstwo Zdrowia, MZ"; WHO, etc. The results for "fake news" were checked for whether the false information on the website had been deemed false. Fake news was defined as information inconsistent with the knowledge and information provided by the WHO, CDC, NHS or national health authorities. The SERPs for the "COVID-19", "Coronavirus" and "SARS-CoV-2" keywords did not contain any websites on alternative medicine; thus, it was not included in further analysis. Each website was classified according to the typology used in other studies [7] : 30 March 2020) that used a governmental domain but were classified as Professional. The first two News (N) websites were analyzed for each of the keywords. The countries were from four continents: Europe (Poland, UK, Spain, Italy, France and Germany), North America (USA), Asia (Singapore) and Australia. Out of these nine countries, six had the highest number of infections as of 27 April 2020: USA, Spain, Italy, France, Germany and UK [11] . It was initially intended to include four countries but eventually extended to include USA, Singapore and Australia and make it more representative and comparable across continents. The European countries had the highest number of COVID-19 cases. Poland was added as the authors' home country. The researchers originally assumed to analyze the first 20 websites returned on each search engine result page (SERP). However, in some countries, keywords such as "fake news" returned only a few SERPs concerning the pandemic. Links to "Wikipedia", "top stories", "ads", the WHO site, paid content and sites requiring registration were excluded. The links on the SERPs that referred to the keywords were copied to a standard Excel spreadsheet and described in detail. The SERPs returned by Google from the "fake news" keyword were included only if they referred to the COVID-19 pandemic. The JAMA score was based on information such as author (authorship), date (currency), financial ownership (disclosure) and references (attribution) [7, 28] . Researchers evaluated each of these four aspects and either awarded a point or not. The JAMA score is the sum of the points awarded to a given website (for information relating to each of the four categories). The evaluated website could therefore receive between 0 and 4 points. In the JAMA evaluation, 1 point is insufficient information, 2-3 points are partially sufficient information and 4 points represent completely sufficient information [7, 28] . Search results were tabulated with numbers and percentages. The COVID-19 epidemiological data for each analyzed country were read from the Worldometer website for the period from 1-27 April 2020 and presented as numbers [11] . The data covered: number of deaths, number of infections, number of SARS-CoV-2 tests performed. A Spearman rank correlation coefficient was used to determine associations between the search results and COVID-19 epidemiological data [29] [30] [31] . Correlation coefficients were calculated between COVID-19 epidemiological data and the frequency of coronavirus articles in the websites, and the frequency of information types in the websites. The correlation coefficient was considered statistically significant at p < 0.05. The JAMA score was used to assess the reliability of each website, where the score considers the author, date, financial ownership and references at 1 point for each. The range of JAMA score values is then from 0 to 4 [25] . The JAMA scores are presented as medians and interquartile ranges (IQR). Statistical analyses were performed with STATISTICA v 13.1 (Dell Inc., 2016, Tulsa, OK, USA). "COVID-19", "Coronavirus", "SARS-CoV-2" and "Fake News" The detailed analysis covered 685 websites in nine countries. The most frequent types were news (47%, n = 324) and governmental (19% n = 131) websites. Governmental pages accounted for the majority of the websites in the United States (35%, 26/74). Australia had roughly similar percentages of news 40% (32/80) and governmental 38.7% (31/80) websites. Commercial websites had substantial shares in Singapore 24.6% (20/81), United States 17.5% (13/74) and Poland 16.4% (12/73). Professional websites had the highest shares in Spain 14.4% (11/76), Germany 18.7% (15/80) and Italy 16.2% (13/80). Scientific journal websites had the largest share in Singapore and the United States (both with 14.8%). Health portals and non-profit websites rarely referred to the 4 keywords, at 2.4% (17/685) and 4.5% (34/685) respectively. Detailed information is presented in Table 1 . News websites were the most common type in six of the nine analyzed countries. In three, Poland, Germany and France, their share exceeded 50% (37/73; 51/80; 59/80). United States and Australia had the highest share of governmental websites. Detailed information is presented in Table 2 . Table 2 . Composition of the search engine result page (SERP) by typology of webpages for keywords: "COVID-19", "Coronavirus", "SARS-CoV-2". Color intensity indicates percentage values-the darker the color, the higher the percentage: up to 20%, >30%, >40% and >50%. G-Government, H-Health Portal, N-News, NP-Non-Profit, P-Professional, C-Commercial, SJ-Scientific journal. Out of the 685 SERPs for the "COVID-19", "Coronavirus", "SARS-CoV-2" and "fake news" keywords, 26.1% (179/685) referred to "COVID-19", 26.2% (180/685) to "Coronavirus" and 25.2% (173/685) to "SARS-CoV-2". Of the 179 SERPs for the "COVID-19" keyword, governmental websites accounted for 33% (59/179) and news websites for 31% (56/179). Of the 180 SERPs for the "Coronavirus" keyword, news websites accounted for 58% (105/180) and governmental websites for 24% (43/180). Of the 173 SERPs for the "SARS-CoV-2" keyword, news websites accounted for 24% (42/173), professional websites for 22% (38/173), scientific journal websites for 18% (31/173) and governmental websites for 15% (26/173). Detailed information is presented in Figure 1a The analysis covered 532 websites that referred to the "COVID-19", "Coronavirus", "SARS-CoV-2" keywords. The sites included the following information: prevention against infection 32.7% (174/532), epidemiological data 31.3% (167/532), symptoms of diseases/infection 26.1% (139/532), diseases/infection risk factors 24% (128/532), transmission routes 23.6% (126/532). There was a large number of websites classified as other, 48% (256/532). They included information such as regulations during the pandemic, services or the economy. The frequency of websites with information on epidemiology ranged from 28% (17/60) for Australia to 42% (25/60) for Germany. For the websites describing preventive measures, the percentage was between 21% (13/61) for Singapore and 50% (30/60) for Poland. Detailed results for each country are presented on Figure 2 . The analysis covered 532 websites that referred to the "COVID-19", "Coronavirus", "SARS-CoV-2" keywords. The sites included the following information: prevention against infection 32.7% (174/532), epidemiological data 31.3% (167/532), symptoms of diseases/infection 26.1% (139/532), diseases/infection risk factors 24% (128/532), transmission routes 23.6% (126/532). There was a large number of websites classified as other, 48% (256/532). They included information such as regulations during the pandemic, services or the economy. The frequency of websites with information on epidemiology ranged from 28% (17/60) for Australia to 42% (25/60) for Germany. For the websites describing preventive measures, the percentage was between 21% (13/61) for Singapore and 50% (30/60) for Poland. Detailed results for each country are presented on Figure 2 . Only 2.6% (14/532) of the websites referred to information on fake news. The research team judged 531 of the 532 websites reliable/correct (based on the scientific knowledge from GIS, NFZ, MZ, WHO, etc.). One website contained questionable information. Details are presented in Supplementary Materials (Table S1) 3.3. Calculating the Frequency of Fake News Referrals from the "COVID-19", "Coronavirus" and "SARS-CoV-2" Keywords Only 2.6% (14/532) of the websites referred to information on fake news. The research team judged 531 of the 532 websites reliable/correct (based on the scientific knowledge from GIS, NFZ, MZ, WHO, etc.). One website contained questionable information. Details are presented in Supplementary Materials (Table S1 ). Of the 153 SERPs that made references to "fake news", 89.5% (137/153) contained information that denied the fake news circulating around the internet and social media. Three websites did not contain any denials, while one had information that had to do with Spanish politics and could not be precisely classified. Most of the 153 pages, 98% (150/153), referred to COVID-19, 34.6% (53/153) referred to preventive measures, and 36% (55/153) contained information that could only be classified as other (regulations during the pandemic, services, the economy, origins of the virus). Detailed information is presented in the Supplementary Files (Table S2 ). In France, Italy, Germany, Poland and Australia, the information returned about fake news on the COVID-19 pandemic usually referred to prevention (45-55%), while in USA and Spain, the percentages were just 7% (1/14) and 6% (1/16), respectively. Details are presented in the Supplementary Files (Table S2 ) and in Figure 3 . Of the 153 SERPs that made references to "fake news", 89.5% (137/153) contained information that denied the fake news circulating around the internet and social media. Three websites did not contain any denials, while one had information that had to do with Spanish politics and could not be precisely classified. Most of the 153 pages, 98% (150/153), referred to COVID-19, 34.6% (53/153) referred to preventive measures, and 36% (55/153) contained information that could only be classified as other (regulations during the pandemic, services, the economy, origins of the virus). Detailed information is presented in the Supplementary Files (Table S2 ). In France, Italy, Germany, Poland and Australia, the information returned about fake news on the COVID-19 pandemic usually referred to prevention (45-55%), while in USA and Spain, the percentages were just 7% (1/14) and 6% (1/16), respectively. Details are presented in the Supplementary Files (Table S2 ) and in Figure 3 . Of the 153 SERPs for the "fake news" keyword, researchers identified 448 pieces of information that referred to fake news on the COVID-19 pandemic (debunked fake news) and grouped them into the 9 most frequent topics: coronavirus was created by peoples Of the 153 SERPs for the "fake news" keyword, researchers identified 448 pieces of information that referred to fake news on the COVID-19 pandemic (debunked fake news) and grouped them into the 9 most frequent topics: coronavirus was created by peoples "miraculous" ways to prevent infection, routes of transmission, underestimating the virus, "harmful" technologies causing the pandemic, children, youths and black people being resilient to coronavirus infections, "effective" treatment methods, local sensations related to the virus, instructions on how to avoid fake news, and others. The most frequent topic was "miraculous" ways of preventing infection, 32% (145/448), followed by local sensations, 13% (56/448) and "effective" treatment methods, 10% (45/448). Detailed data are presented in Table 3 while Table 4 includes examples of debunked fake news. "Please, be informed that according to the special act on coronavirus, your funds will be transferred to the Polish Central Bank. Log in in order to keep your 1000 PLN". Singapore 1. Millions of Facebook users continue to be exposed to coronavirus misinformation: some of the most dangerous falsehoods had received hundreds of thousands of views, including claims like "black people are resistant to coronavirus" and "Coronavirus is destroyed by chlorine dioxide". A message circulating on messaging platforms and social media claiming that an Enterprise Singapore safe-distancing ambassador had fined someone for sitting on a seat that was marked out as part of safe-distancing measures is false. Des "petites bombes" pour tuer le virus vont être lâchées à 3 heures du matin par avion ("'Small bombs' to kill the virus will be dropped at 3 am by plane"). Italy 1. l'italia si trasformi a terra conquista tramite il fenomeno "5G" che va contenuto e controllato controllato (Italy is transforming itself into a land conquered through the technology "5G" that must be contained and controlled). ( . . . ) use of hand sanitizers contributed to cancer. The latest coronavirus has been found in energy drinks. Some of the content was very long; therefore, the omitted text was marked as: "( . . . )". The research team attempted to find associations between the frequency of articles on coronavirus in the different types of websites and epidemiological data for countries, i.e., number of tests performed, number of infected people or number of deaths. The Spearman's correlation coefficients were high (0.5-0.6); however, the low number of countries made the results statistically insignificant. Another analysis investigated the association between the frequency of information types and epidemiological data. The frequency of different content types did not correlate with data on the number of tests, number of infected people or deaths. There was an association between information on preventive measures and the number of people infected. Australia and Poland had low numbers of infections and had the largest amount of information about prevention. Detailed data are presented on Figure 4 . The research team attempted to find associations between the frequency of articles on coronavirus in the different types of websites and epidemiological data for countries, i.e., number of tests performed, number of infected people or number of deaths. The Spearman's correlation coefficients were high (0.5-0.6); however, the low number of countries made the results statistically insignificant. Another analysis investigated the association between the frequency of information types and epidemiological data. The frequency of different content types did not correlate with data on the number of tests, number of infected people or deaths. There was an association between information on preventive measures and the number of people infected. Australia and Poland had low numbers of infections and had the largest amount of information about prevention. Detailed data are presented on Figure 4 . The information about fake news that referred to the COVID-19 pandemic was dominated by preventive measures. Researchers checked whether the frequency of fake news about prevention was correlated with epidemiological data and observed that the higher the number of tests, the higher the frequency of information on fake news referring to prevention. On the other hand, it was found that the higher the number of infections, the lower the amount of information on fake news referring to prevention. The scatter plots on Figure 5 present the detailed data. The information about fake news that referred to the COVID-19 pandemic was dominated by preventive measures. Researchers checked whether the frequency of fake news about prevention was correlated with epidemiological data and observed that the higher the number of tests, the higher the frequency of information on fake news referring to prevention. On the other hand, it was found that the higher the number of infections, the lower the amount of information on fake news referring to prevention. The scatter plots on Figure 5 present the detailed data. Of the 532 SERPs for the "COVID-19", "Coronavirus" and "SARS-CoV-2" keywords, only 3% (14/532) referred to information on testimonials or celebrities and less than 1% (2/532) to religion (Supplementary Files, Table S1 ). Of the 153 SERPs for the "fake news" keyword, 1% (18/153), 8% (13/153) and 2% (3/153) referred to testimonials, celebrities and religion, respectively (Supplementary Files, Table S2 ). Table 5 presents examples of information on celebrities, religion and testimonials. Table 5 . Examples of whether a testimonial, celebrity, religion was mentioned. Poland testimonial "Jason Hargrove, a bus driver from Detroit, was irritated with passengers coughing all the time and posted a video in which he called for hygiene during the pandemic. He died 11 days later due to the coronavirus." Singapore NA UK testimonial "A museum boss says he had hallucinations filled with "visions of snakes" after being struck down by coronavirus." 3.6. Characterization and Analysis of Online Information Returned by Google Based on the "COVID-19", "Coronavirus", "SARS-CoV-2" and "Fake News" Keywords about Celebrities, Religion and Testimonials Of the 532 SERPs for the "COVID-19", "Coronavirus" and "SARS-CoV-2" keywords, only 3% (14/532) referred to information on testimonials or celebrities and less than 1% (2/532) to religion (Supplementary Files, Table S1 ). Of the 153 SERPs for the "fake news" keyword, 1% (18/153), 8% (13/153) and 2% (3/153) referred to testimonials, celebrities and religion, respectively (Supplementary Files, Table S2 ). Table 5 presents examples of information on celebrities, religion and testimonials. Table 5 . Examples of whether a testimonial, celebrity, religion was mentioned. Poland testimonial "Jason Hargrove, a bus driver from Detroit, was irritated with passengers coughing all the time and posted a video in which he called for hygiene during the pandemic. He died 11 days later due to the coronavirus". Singapore NA UK testimonial "A museum boss says he had hallucinations filled with "visions of snakes" after being struck down by coronavirus". Some of the content was very long; therefore, the omitted text was marked as: "( . . . )". NA-not applicable. The median JAMA score for all SERPs is shown in Figure 6 . The Singapore and USA SERPs have a significantly higher JAMA score than any other SERPs. The scores for Australia, Poland and UK, Germany, Italy and France were comparable. Ramazan 2020 in den @Westfalenhallen nicht stattfindet." France celebrity ""L'intégralité des cours manqués à cause du COVID-19 seront donc rattrapés pendant les vacances d'été". La nouvelle est apparue sous la forme d'une capture d'écran d'un prétendu tweet d'Emmanuel Macron. Il s'agit évidemment d'un faux, puisque le compte à l'origine de la publication est @EmmanuelMecron" Italy NA Australia religion "Chinese people are not converting to Islam because of the outbreak." Some of the content was very long; therefore, the omitted text was marked as: "(…)". NA-not applicable. The median JAMA score for all SERPs is shown in Figure 6 . The Singapore and USA SERPs have a significantly higher JAMA score than any other SERPs. The scores for Australia, Poland and UK, Germany, Italy and France were comparable. Daily messages that inform citizens about the current epidemiological situation and recommend self-isolation cause social concerns and anxiety. Meanwhile, the lack of reliable information encourages people to explore the internet. Such a situation may foster conspiracy theories, such as ones about the coronavirus being created by someone in order to limit freedom and manipulate people [32] . Many governments take action and try to be informative by setting up official websites or delegating such tasks to other official institutions (such as the Robert Koch Institute in Germany). Our results show that information Characterization and Analysis f Websites Typology by Google Based on the "COVID-19", "Coronavirus", "SARS-CoV-2" and "Fake News" Keywords Daily messages that inform citizens about the current epidemiological situation and recommend self-isolation cause social concerns and anxiety. Meanwhile, the lack of reliable information encourages people to explore the internet. Such a situation may foster conspiracy theories, such as ones about the coronavirus being created by someone in order to limit freedom and manipulate people [32] . Many governments take action and try to be informative by setting up official websites or delegating such tasks to other official institutions (such as the Robert Koch Institute in Germany). Our results show that information on coronavirus is most frequently posted on information (News) and governmental (dominated by USA and Australia) sites. Almost all of the information was true. The words "COVID-19" and "Coronavirus" were more common on governmental and news websites, while "SARS-CoV-2" was more typical for scientific journals and professional websites. The results obtained in our study correspond to the results of the team of Okan O. et al. conducted in Germany in the period from 30 March 2020 to 7 April 2020 in the form of an online survey, where issues related to COVID-19 were discussed [33] . The study concerned the respondents' assessment of the ease of access to reliable content on COVID-19. Researchers have confirmed the importance of local authorities in preventing the spread of the infopandemic [33] . Information related to the "COVID-19", "Coronavirus", "SARS-CoV-2" keyword searches referred to the following topics: preventive measures against infection, epidemiological data, symptoms of the diseases/virus, infection risk factors, transmission routes. A large share (48%, 256/532) of the information that was classified as other and included issues like regulations during the pandemic, services or the economy, referred also to the etiology of SARS-CoV-2. There were notable differences between the frequency of information on websites describing transmission routes: from 5% (3/60) in Australia to 52% (31/60) in Poland. Importantly, countries with the highest number of infections (Italy and France) had a low share of websites about symptoms. In Singapore, most content on COVID-19 referred to epidemiological data while Australia, Italy and France had the highest share of sites with preventive measures. This may be credited to actions taken by governments as a result of the growing number of infections. These results for Italy are in line with the work of Rovetta and Bhagavathula (2020) who found that the 5 most important searches related to health in Italy were: face masks, amuchina (disinfectant gel), new virus symptoms, health bulletin and vaccines against coronavirus [34] . Most of the countries covered by this analysis reported their first cases of COVID-19 during the last week of January 2020. Poland was the last to report its first case, on 4 March 2020. The disease spread at a different pace in each country, which was related to various factors such as the regional structure of the demography, density of the population, social mobility or government policy. The number of infected people around the world increased threefold between 1 and 27 April 2020 from 940.5 thousand to over 3 million. The largest growth from 220 thousand to over 1 million was reported in the United States. On the 1 April, at least 100 thousand infections were identified in three of the analyzed countries: United States, Italy and Spain. In three weeks, this list has expanded to six countries, adding France, Germany and UK. The pandemic had the lowest pace in Australia, which on 1 April reported an approximately 30% increase from 5048 to 6720 infections. Meanwhile, the number of COVID-19 related deaths around the world surged from 48.5 thousand to 211.5 thousand, with the United States leading the statistics (6394 to 56, 796) . At the end of April 2020, more than 20,000 deaths were reported by Italy, Spain, France and UK. The COVID-19 mortality rate was 6.9% on 27 April 2020 with the highest rates in France (14.3%), Italy (13.5%) and UK (13.4%), and the lowest in Singapore (0.1%) and Australia (1.2%) [13] . It is notable that there were significant differences in this result between countries when it comes to the typology of websites and the information they contained. This analysis showed that Australia and Poland had the lowest infection rates per 1 m citizens (264 and 311, respectively) along with the largest share of content about prevention. This suggest that the effectiveness of preventive measures in both countries was relatively high. An important fact is that almost 90% (137/153) of information about fake news in the COVID-19 pandemic emphasized that it referred to fake news. Three news reports (1 in Australia and 2 in Italy) could not be reliably classified. Most commonly, SERPs for the "fake news" keyword described examples of fake news on the Internet and social media. Countries with the highest number of infections (Italy, Germany, USA, France) also had the highest share of fake news about the coronavirus pandemic. In a similar period (31 December 2019-30 April 2020), a study on fake news on the Internet was conducted in Italy. This study used the BuzzSumo program and examined social media: Facebook, Pinterest, Reddit and Twitter. As with our study, researchers manually rated each post for its truthfulness [16] . The WHO has published 19 explanations and clarifications to the most frequent fake news on COVID-19 and created a simple infographic [35] . The French media reported in February 2020 that the government organized a meeting of major social media actors attended by representatives of Google, Qwant, Facebook, Twitter, Instagram, TikTok and LinkedIn and numerous government officials. The objective was to facilitate cooperation between government and online media [36] . The strategy implemented by the French seems to be reflected in these results as most of the information referred to prevention or regulations during the pandemic. The 20 SERPs for the "fake news" keyword included websites that described and corrected 152 fake news pieces. It should be noted that the results indicate a low prevalence of fake news about the coronavirus pandemic in the Google search engine during the analyzed period. This is in contrast to the results of Heidi Oi-Yee Li et al., who found that more than a quarter of the most viewed YouTube videos on COVID-19 contained misleading information, reaching millions of viewers worldwide [37] . This raises a question of whether Google search provides more accurate information and minimizes the spread of misinformation. It would suggest its significant role in managing the COVID-19 pandemic. Huynh Dagher S et al. used similar tools in their study to determine the influence of the media coverage and government policies on Google searches of the symptoms of COVID-19 in the six countries: the United States, the UK, France, Italy, Spain and Germany. The keywords selected for this purpose required translation into the respective national languages, which could then have influenced the results. Dagher et al. concluded that that Internet users were influenced by the media coverage and governmental policies [38] . Zeng K et al. covered six countries: the United States, Spain, Italy, Taiwan, South Korea and Singapore, and concluded that early digital intervention had a strong correlation with the successful containment of COVID-19 [39] . There were already some initiatives aimed at fighting fake news even before the pandemic. The European Commission set up a high-level group of experts ("the HLEG") in 2018 that started working on a report about disinformation and the principles of forwarding information [40] . The UK published its own report on fake news [41] . Another British initiative referred to creating the International Grand Committee on Disinformation and "Fake News" that involved representatives from Argentina, Belgium, Brazil, Canada, France, Ireland, Latvia and Singapore. As a result, the "Principles of the Law Governing the Internet" declaration was signed in November 2018 [42] . Singapore implemented in June 2019 the "Protection from online falsehoods and manipulation" act, making it illegal to publish false information and implementing penalties. The fight against fake news was intensified after the outburst of the pandemic [43] and these results seem to reflect that. A number of institutions and social media companies such as Facebook or Twitter implemented measures during the pandemic aimed at preventing fake news from spreading. Examples include adding automatically WHO links to every post about the coronavirus. WhatsApp decided to limit forwarding of messages in order to combat fake news [6] . Considering the results of the Abd-Alrazaq et al., 2020 [14] , which identified major topics on COVID-19 published by Twitter users, such actions and measures are crucial when it comes to combating spread of false information. There were over 927,000 recoveries from COVID-19 around the world up to 27 April 2020 and 211,500 deaths which accounted for 81.2% of closed cases [11, 13] . The highest rate was reported in Singapore (98.7%), Australia (98.5%) and Germany (94.9%) [11, 13] . However, as countries employed different policies of infections control and reported various number of tests performed, the real data may be different. The highest number of tests per 1 m was performed in Italy (over 29,000) with the lowest in France (7.100) [11, 13] . The study results suggest that the higher the number of SARS-CoV-2 infections is in a country, the lower is the amount of information on fake news referring to prevention. On the other hand, the higher was the number of tests in a country, the larger was the amount of information on fake news related to prevention. The higher frequency of articles indicating false information online is a positive phenomenon as it indicates effectiveness of actions taken by policy makers against fake news and disinformation. The intention of this study was to use the methodology described by Arif et al. [7] with regards to information on COVID-19. The results suggest that investigating information from the Google search engine (SERP) may be a useful tool for public health information screening in cyberspace. Along with an increasing number of infections, there is a growing amount of information on prevention, treatment and consequences of a disease published by governmental, health, science or media organizations. This is also true for unreliable content circulated to the public [6, 7, 14, 32, 34, 44] . The authors modeled the methodology used in the study to investigate vaccinations [7] . The research team found that it may also be used for other purposes in analyzing online information such as the global COVID-19 pandemic. This study provides a broader view on the issue of information available online about the pandemic and covers 4 continents, which makes it more representative. Most of the countries took measures aimed at fake news on COVID-19 [24, 45] . However, not all of them were equally effective. Tapia investigated fake news on COVID-19 in the Dominican Republic and indicated that the political crisis and turmoil caused by "fake news" hampers the effective response of the authorities [46] . It is also necessary that major media take responsibility for providing proper and correct content. Journalists must be aware of their important role as false or inaccurate information may have serious consequences. Last but not least, health professionals should cooperate with mass media to counter harmful myths. Effective cooperation and communication between all involved parties seems crucial in eliminating fake news [47, 48] . Thus, it is necessary to continue and expand measures and actions taken at a national and global level aimed at fighting disinformation. The public health should play a crucial role in this area. First of all, this is a qualitative study that lacks representativeness and thus should not be generalized [27] . Nevertheless, its wide scope may be considered a strength. It covered countries from 4 continents: Europe, North America, Asia and Australia. Six out of the 9 analyzed countries had the highest number of infections as of 27 April 2020. This research referred to crucial elements of the healthcare system and public health such as social exposure to health risks from coronavirus fake news. The content analysis during the pandemic was very wide and allowed us to thoroughly investigate the issue of fake news. Another limitation is cultural differences between the societies of individual countries, which were not taken into account. The diversity of topics related to the pandemic in different countries and the way they are communicated shows the importance of differences in the perception of reality. Such differences among inhabitants of different regions despite the use of the same language were already indicated in a study by Hughes et al. in 2021 [49] . The SERPs were classified according to 7 different topics, judged true or false, and each website was evaluated according to 17 elements. The SERPs for the "fake news" keyword were checked as to whether the fake news was corrected and explained. The study was conducted in a short period of time that covered the pandemic peak and time in which governments implemented various measures. The team analyzed 685 SERPs. Although Arif N. et al., 2018 , investigated the first 100 websites for each country [7] , this analysis covered an entirely new issue for the world; thus, the original assumptions had to be changed. The lower number of SERPs than originally planned may stem from government policies aimed at fighting disinformation. UK had 61 SERPs, Spain 76, United States 74 and Poland 73. However, researchers believe that the 685 SERPs still make it a reliable result. A similar study using SERPs was also carried out in March 2020. Here, too, a review of the first 30 SERPs was assumed, using previous research results [50] on Internet usage, showing that 90% of users are limited to the first 30 addresses [51] . The purpose of this study was to evaluate online patient educational material on COVID-19 for understandability. Hernández-García and Giménez-Júlvez analyzed 80 weblinks in total on COVID-19 for the United States and Spain [44] . The important limitation is the location from which the study was conducted. The computer's location may affect the results; however, the research team implemented measures in order to mitigate a risk of biased results. The authors did not use a VPN during research, which made the results more neutral. It is also essential there were not any adds-on in the browser. Another limitation of this study is that we only looked at webpages and did not investigate social networks. In addition, we used the same, neutral, keywords without taking into account potential differences in the most searched terms used in national languages. It is likely that users could find more biased information by using more negative or positive search terms. The study may also be subject to information bias that may have occurred during data collection (content analysis). Due to the nature of the study, the fundamental error that could have occurred was a misclassification bias and observer/interviewer bias. However, it is acceptable that most of research is subject to some degree of misclassification [52] . The information on celebrities in relation to COVID-19 may be subjective as it frequently referred to health policy decision makers; thus, it was up to Google algorithms to decide whether information was related to a celebrity or something else, such as, for example, the economy. It did not, however, influence other results. The researchers also attempted to find associations between the frequency of new articles about the coronavirus on different types of websites and epidemiological data for the countries; nevertheless, the low number of analyzed countries in terms of statistical requirements made it impossible to find statistical significance. When undertaking the study, the researchers assumed that there would be more fake news in the first months of the pandemic than there actually were. Due to the fact that the study was conducted in the initial period of the pandemic, it would be worth repeating it using the same methodology. Currently, fake news related to COVID-19 has been the subject of number of publications. Their main purpose was usually identifying and controlling fake news [53] [54] [55] [56] . The analyzed content about COVID-19 returned by Google during the pandemic showed that governments in the analyzed countries took effective measures in fighting fake news during the pandemic. Most of the published information was available on news or governments sites, referred to prevention, epidemiological data or disease symptoms. There were differences between continents when it came to the types of information available online: Asia was dominated by epidemiological data, Western Europe and Australia by prevention, and North America and Central Europe by risk factors. The COVID-19 pandemic information, including fake news, rarely made reference to celebrities, religion or testimonials. The JAMA score was comparable in most of the analyzed countries, except for Singapore and USA, where it was higher. Although the first 20 SERPs from the COVID-19, coronavirus and SARS-CoV-2 keywords contained true information, it is inevitable that false information did make its way to the Internet when analyzed more deeply. Most commonly, SERPs for the "fake news" keyword described examples of fake news on the Internet and social media. In countries with the highest number of tests, a higher frequency of information on fake news referring to prevention was found. The higher the number of SARS-CoV-2 infections, the lower the amount of information on fake news referring to prevention against infection with the virus. Supplementary Materials: The following supporting information can be downloaded at: https: //www.mdpi.com/article/10.3390/ijerph19031491/s1, Table S1 : Types of information from Google using the "COVID-19", "Coronavirus" and "SARS-CoV-2" keywords, by country (n = 532); Table S2 : Types of information about COVID-19 returned by Google using the "fake news" keyword, by country (n = 153). The authors declare no conflict of interest. The science of fake news Fake news, alternative facts, and things that just are not true can science survive the post-truth era? A trust model for spreading gossip in social networks: A multi-type bootstrap percolation model Evaluating the fake news problem at the scale of the information ecosystem Information Management in Healthcare and Environment: Towards an Automatic System for Fake News Detection Going viral: Doctors must tackle fake news in the covid-19 pandemic Fake News or Weak Science? Visibility and Characterization of Antivaccine Webpages Returned by Google in Different Languages and Countries Vaccine hesitancy and (fake) news: Quasi-experimental evidence from Italy. Health Econ Fake news and dental education Fake news and patient-familyphysician interaction in critical care: Concepts, beliefs and potential countermeasures COVID-19 Coronavirus Pandemic Methods for Estimating the Case Fatality Ratio for a Novel, Emerging Infectious Disease Own Calculations Based on Data on COVID-19 (Coronavirus) by Our World in Data Top Concerns of Tweeters During the COVID-19 Pandemic: Infoveillance Study Advice for the Public: Coronavirus Disease (COVID-19) Fake News and Covid-19 in Italy: Results of a Quantitative Observational Study Report-1; World Health Organization Report-11; World Health Organization WHO. Naming the Coronavirus Disease (COVID-19) and the Virus That Causes It Use ICD Codes for COVID-19 Disease Outbreak International Guidelines for Certification and Classification (Coding) of Covid-19 as Cause of Death Based on ICD International Statistical Classification of Diseases; World Health Organization COVID-19) Situation Report-51; World Health Organization Report-13; World Health Organization Infodemic" and Emerging Issues through a Data Lens: The Case of China Infodemiology: Tracking Flu-Related Searches on the Web for Syndromic Surveillance Infodemiology and Infoveillance: Framework for an Emerging Set of Public Health Informatics Methods to Analyze Search, Communication and Publication Behavior on the Internet Standards for reporting qualitative research: A synthesis of recommendations Evaluation of Internet Information about Lingual Orthodontics Using DISCERN and JAMA Tools Understanding the Community Risk Perceptions of the COVID-19 Outbreak in South Korea: Infodemiology Study Investigating the Prevalence of Reactive Online Searching in the COVID-19 Pandemic: Infoveillance Study Public Interest in Immunity and the Justification for Intervention in the Early Stages of the COVID-19 Pandemic: Analysis of Google Trends Data Dangerous Messages or Satire? Analysing the Conspiracy Theory Linking 5G to COVID-19 through Social Network Analysis Coronavirus-Related Health Literacy: A Cross-Sectional Study in Adults during the COVID-19 Infodemic in Germany COVID-19-Related Web Search Behaviors and Infodemic Attitudes in Italy: Infodemiological Study. JMIR Public Health Surveill. 2020, 6, e19374 COVID-19) Advice for the Public: Myth Busters; World Health Organization Comment le Gouvernement et les Réseaux Sociaux Luttent Contre les "Fake News YouTube as a source of information on COVID-19: A pandemic of misinformation? The Influence of Media Coverage and Governmental Policies on Google Queries Related to COVID-19 Cutaneous Symptoms: Infodemiology Study The Use of Digital Tools to Mitigate the COVID-1 Pandemic: Comparative Retrospective Study of Six Countries European Commission. A Multi-Dimensional Approach to Disinformation. Report of the Independent High Level Group on Fake News and Online Disinformation. Bruxelles. 11 House of Commons. Disinformation and 'Fake News': Interim Report. Digital, Culture, Media and Sport Committee Parliamentarians from Across the World Sign Declaration on the 'Principles of the Law Governing the Internet'. London Republic of Singapore Government Gazette Acts Supplement Assessment of Health Information About COVID-19 Prevention on the Internet: Infodemiological Study. JMIR Public Health Surveill. 2020, 6, e18717 An Infodemiological Study on Novel Coronavirus (COVID-19) in South Korea: Conversations and Medical News Frames on Twitter Novel Coronavirus Disease (COVID-19) and Fake News in the Dominican Republic 2019-nCoV, fake news, and racism Infodemic and the spread of fake news in the COVID-19-era Cultural Variance in Reception and Interpretation of Social Media COVID-19 Disinformation in French-Speaking Regions Readability of online patient education material for the novel coronavirus disease (COVID-19): A cross-sectional health literacy study Understanding patterns of COVID infodemic: A systematic and pragmatic approach to curb fake news The Contribution of Informatics to Overcoming the Covid-19 Fake News Outbreak by Learning to Navigate the Infodemic Detecting COVID-19-Related Fake News Using Feature Extraction. Front A Deadly Infodemic: Social Media and the Power of COVID-19 Misinformation