key: cord-0996539-rhrdsds1 authors: Jarynowski, A.; Semenov, A.; Kaminski, M.; Belik, V. title: Mild Adverse Events of Sputnik V Vaccine Extracted from Russian Language Telegram Posts via BERT Deep Learning Model date: 2021-06-22 journal: nan DOI: 10.1101/2021.06.14.21258875 sha: 02a91ada46a3cd45f6d600d09f28dabbbfab7236 doc_id: 996539 cord_uid: rhrdsds1 Background: There is a limited amount of data on the COVID-19 vector vaccine Gam-COVID-Vac (Sputnik V) safety profile. Previous infodemiology studies showed that social media discourse could be analyzed to assess the most concerning adverse events (AE) caused by drugs. Objective: We aimed to investigate mild AEs of Sputnik V based on a participatory trial conducted on Telegram in the Russian language. We compared AEs extracted from Telegram with other limited databases on Sputnik V and other COVID-19 vaccines. We explored symptom co-occurrence patterns and determined how counts of administered doses, age, gender, and sequence of shots could confound the reporting of AEs. Materials and Methods: We collected a unique dataset consisting of 11,515 self-reported Sputnik V vaccine AEs posted on the Telegram group, and we utilized natural language processing methods to extract AEs. Specifically, we performed multi-label classifications using the deep neural language model BERT DeepPavlov, which we pre-trained on a Russian language corpus and applied to the Telegram messages. The resulting AUC score was 0.991. We chose symptom classes that represented the following AEs: fever, pain, chills, fatigue, nausea/vomiting, headache, insomnia, lymph node enlargement, erythema, pruritus, swelling, and diarrhea. Results: The results of the retrospective analysis showed that females reported more AEs than males (1.2-fold, P<.001). In addition, there were more AEs from the first dose than from the second dose (1.13-fold, P<.001), and the number of AEs decreased with age ({beta} = .05 per year, P<.001). The results also showed that Sputnik V AEs were more similar to other vector vaccines (132 units) compared with mRNA ones (241 units) according to the average Euclidean distance between the vectors of AE frequencies. Elderly Telegram users reported significantly more (5.6-fold on average) systemic AEs than their peers, according to the results of the phase III clinical trials published in The Lancet. However, the AEs reported in Telegram posts were consistent (Pearson correlation r=.94, P=.02) with those reported in the Argentinian post-marketing AE registry. Conclusion: After receiving the Sputnik V vaccination, Telegram users complained about pain (47%), fever (47%), fatigue (34%), and headache (25%). The results showed that the AE profile of Sputnik V was comparable with other COVID-19 vaccines. Examining the sentinel properties of participatory trials (which is subject to self-reporting biases) could still provide meaningful information about pharmaceutics, especially if only a limited amount of information on AEs is provided by producers. The current COVID-19 pandemic is one of the most critical global health problems. The main strategies for its mitigation involve both non-pharmaceutical interventions (e.g., testing and contract tracing) and up-to-date anti-COVID-19 treatments. However, the most promising remedy has been vaccines that have effectively prevented severe COVID-19 outcomes. In addition to novel mRNA vaccines, vector vaccines have been developed. One of the first was Gam-COVID-Vac (Sputnik V), which is a viral two-dose vector vaccine based on two human adenoviruses. Each dose contains a different vector: rAd26 and rAd5. This vaccine was developed by the Gamaleya Research Institute of Epidemiology and Microbiology. Sputnik V contains a gene that encodes SARS-CoV-2's spike (S) protein [1] . To date, two reports of clinical trials have been published. In the first study, of I/II phases involved a total of 76 participants, who were included in the safety analysis [2] . The report on phase III included detailed descriptions of serious and rare adverse events (AE) as well as mild AEs described in individuals [3] older than 60 years. The overall frequency of AEs was mentioned without complete characteristics of the safety profile, such as the co-occurrence of AEs. Mild AEs are common among all vaccines. Extensive fact sheets on AEs, as well as possible adverse reactions, were provided for vaccines trialed under US (FDA -Food and Drug Administration), UK (MHRA -Medicines and Healthcare products Regulatory Agency), or EU (EMA -European Medicines Agency) regulator bodies, which was not the case with Sputnik V. Until 20 March 2021, of the almost 30 million doses manufactured, approximately eight million doses [4] of COVID-19 immunization have been administered in a single dose in Russia [5] . Moreover, the Russian Federation has signed contracts with dozens of countries to deliver 1.4 billion doses at less than 7 EUR per dose for international buyers [6] . Therefore, there is an emerging need to update the information on Sputnik V's safety profile using post-marketing surveillance. Because a registry of AEs after vaccination with Sputnik V is difficult to access, social media discourse may be an alternate source of information on AEs. An increasing number of studies have analyzed English-language social media in the context of vaccinations [7] or vaccine-prevented infectious disease [8] . However, only a few similar studies on Russian social media users have been published [9, 10] . Accounts of adverse reactions to drugs have been widely extracted from social media [11] in the context of mining consumer reviews on the Internet [12] . To date, most of these studies processed data collected from Twitter [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] . Although social media platforms such as Twitter and Facebook are used in Russia, the most popular platforms are VK.com, which has about 97 million active monthly users [23] , and Telegram Messenger, which is ranked second in the Russian Appstore, having 500 million active users overall. Developed in Russia, these platforms are much more popular than alternatives such as Facebook or Twitter [23, 24] . Most previous studies on social media vaccine discourse have focused on the personal beliefs of users. For example, Wang et al. developed a framework to detect vaccine AEs mentioned by Twitter users [25] . However, to date, no study has analyzed social media discourse on non-severe AEs in response to COVID-19 vaccines. In the present study, we collected social media (a Telegram group in the Russian language) data to bridge the gap in information on the most prevalent AEs involving Sputnik V. We focused on the most common AEs and established which were the most prevalent, their co-occurrence, and their associations with users' characteristics [26] . Finally, we compared the AE profile of Sputnik V with those of other approved COVID-19 vaccines. The dataset analyzed in our study was collected retrospectively from the Telegram group 1 , "Sputnik_results". The data contained no personal information, and the analysis was performed according to the Terms of Service of the platform [27] . Our analysis was completely anonymous and performed in aggregated form. No possible harm to Telegram users was identified. Therefore, the study did not require ethical committee approval. Originally, Telegram was aiming to provide secure communication, but later functionality was expanded; it added support of public channels, groups, video calls, and many other features. Telegram groups may be public or private. If a group is public, it may be accessed via the Telegram search engine, and every user may read all its content. A main priority claimed by Telegram is security; users' data are not disclosed, and only the user's screen name and picture are shown to the public. The largest Telegram channels have millions of subscribers. The description of the "Sputnik_results" public group states that its main aim is to collect information on AEs regarding the Sputnik V vaccine. Telegram users may post a description of their symptoms. Moderators of the group oversee the messages and verify that they contain only descriptions of AEs; otherwise, the message is deleted. In the present study, we collected all messages from the "Sputnik_Results" group using Python Telegram Client telethon [28] . We saved only text messages that were posted in the group; users' personal details were not extracted. In total, we collected 18,833 messages. After filtering messages that contained only pictures, 11,515 messages remained. The first message was sent on 09 December 2020, and the most recent message was sent on 17 April 2021. The dataset contained 25,660 unique lowercase words. 1 https://t.me/Sputnik_results The gold standard used to identify adverse reactions is the MedDRA System Organ Class (SOC), which is applied in the EU (EudraVigilance 2 ), the US (VAERS 3 ), and the UK (MHRA Yellow Card scheme 4 ). However, the system uses a specialized medical vocabulary. In our study, because users of social media communicated in colloquial language [11] , we chose a simplified FDA classification system [29] [30] [31] that was subdivided into two groups: local reactions (i.e., redness, swelling, and pain at the injection site) and systemic reactions (i.e., fever, fatigue, headache, chills, nausea/vomiting, diarrhea, new or worsening muscle pain, and new or worsening joint pain). Moreover, muscle pain, joint pain, and pain at the injection site were categorized as a single class. However, we added the classes of pruritus, enlarged lymph nodes, and insomnia, which are common adverse reactions to anti-COVID-19 vaccines. The final list of 12 classes of symptoms of mild AEs, which were based on subjective experiences of a potential health issue, is provided in Table 2 . Labeling. We utilized the LabelStudio data labeling tool [32] to label the dataset. We randomly sampled 1,000 messages in the dataset, which were labeled by three raters who were native Russian speakers. The raters labeled each occurrence of an AE in the messages, thus making the dataset suitable for named entity recognition (NER) tasks. Because of such labeling and the existence of different descriptions of the same AEs in multiple sentences, we augmented the dataset by splitting each message into sentences. The resulting dataset contained 4,579 entities. Model architecture. Each message in our dataset could have included multiple AEs. We therefore adopted a multi-label text classification scheme. A formal definition of multi-label classification is as follows. Consider a dataset, where is the i-th observed variable for the dataset of = { , is the corresponding set of labels for the i-th element. Our goal was to learn a ∈ mapping , where , is the set of predicted classes, and is a vector of parameters. To find = , θ ( ) θ 4 https://yellowcard.mhra.gov.uk the vector of optimal parameters , we needed to minimize the loss function between the actual θ , ( ) and predicted classes. Multiple machine learning methods may be applied to support multi-label classification. In the case of artificial neural networks (ANN), the activation function of the last layer of the ANN is set to be a sigmoid, and binary cross-entropy loss is used. Because of the recent success of ANNs, specifically transformers, in text analysis tasks, we adopted a deep bidirectional transformer architecture (BERT) to perform our multi-label classification task [33] . We utilized a pre-trained BERT model for the Russian language DeepPavlov [34] . We tuned the last layer of the model, which consisted of 12 sigmoid neurons. As a baseline, we used a long-short term memory (LSTM) ANN, which consisted of embedding as the first layer and one LSTM layer (100 cells), dropout (p = . 2) and a subsequent multi-label dense layer with sigmoid as the activation function. Model evaluation. We trained the BERT and LSTM models using a stratified k-fold validation scheme where k = 5. Because the classes were imbalanced, we utilized an up-sampling strategy; that is, underrepresented classes were up-sampled in the training dataset. The testing set distribution was not modified. Table 1 displays the evaluation results. Precision and recall were calculated for both micro-and macro-averaged aggregations [35] . As shown in Table 1 , precision and F1 scores were reported for thresholds equal to 0.5. We utilized a computer with a Tesla T4 GPU to train the models. Table 1 shows that BERT outperformed the LSTM model by a large margin. We therefore chose the BERT model and trained it on 95% of the data; in this case, it returned a micro-averaged accuracy of 0.94 and an area under the ROC curve (AUC) score of 0.991. 6 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 22, 2021. ; Regarding gender, age, and dose number (if available), we used counts of corresponding abbreviations and regular expression matching because the administrators of the group had provided detailed instructions for the reporting of this information. Reactogenicity assessment based on opt-in civic surveillance was performed to obtain results of clinical importance (similar to endpoints in trials). The volume peak of the self-reports corresponded to the time at which vaccinations were speeded up ( Figure 1 ). Moreover, after three months of vaccinations (the end of February 2021), the popularity of self-reporting started to decrease despite the increasing vaccination roll-out. However, the Pearson correlation coefficient between doses administered until February 28 was very high (r = .7, P<.001) and the subsequent count of administered doses increased, while reports on AEs decreased ( Figure 1 ). (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 22, 2021. ; https://doi.org/10.1101/2021.06.14.21258875 doi: medRxiv preprint Our analysis revealed that fever and generalized pain were the most commonly reported AEs (Table 2, Figure 2 ). Injection site irritations (local reactions) were in the order of magnitude less likely to be reported than fever and pain (systemic reaction). Gastric symptoms (especially diarrhea, with a frequency of 0.6% per report) were less likely to be reported than the average prevalence among the general population (1-5% for diarrhea). Figure 2 : Frequency (%) of given AE classified to be reported by users. Symptoms: orange -systemic; green -local; yellow -gastric 8 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 22, 2021. ; https://doi.org/10.1101/2021.06.14.21258875 doi: medRxiv preprint Gender was reported by 3,992 females and 2,762 males. On average, females reported 2.5 AEs (σ = 1.79), and males reported 2.1 AEs (σ = 1.64). Females reported statistically significantly more AEs (P = 2 · 10 −16 ) according to the results of a Mann-Whitney U test (Table 3 ). AEs in response to other anti-COVID-19 vaccines have been found to depend on whether the vaccination was the first or the second dose (if applicable). For instance, AEs in response to mRNA vaccines have tended to be stronger with the second dose [30, 31, 37] . In contrast, AEs involving vector 9 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 22, 2021. ; https://doi.org/10.1101/2021.06.14.21258875 doi: medRxiv preprint vaccines have tended to be milder with the second dose [38] . Regarding the Sputnik V vaccine, this difference might be because a different vector is used in each dose, which might lead to different reactions. Among the self-reports, 4,174 described AEs after the first dose, 1,251 after the second dose, and 3,049 described AEs after both doses. It is also possible that the users did not receive the second dose because of contraindications or just lost interest in reporting. Here, we considered only reports that discussed the first and second doses separately. On average, there were 2.2 (σ = 1.80) AEs for the first dose and 1.9 (σ = 1.69) AEs for the second dose. According to the results of the Mann-Whitney U test, there were statistically significantly more AEs after the first dose (P = 1.7 · 10 −5 ) ( Table 3) . To quantify the co-occurrence of symptoms, we calculated Spearman's rank correlation coefficients between each pair of classified symptoms. We observed systemic, local, and gastric clusters ( Figure 4) . We also provided a network representation in which vertex size represents symptom prevalence and edge width represents co-occurrence as measured by the correlation coefficient. Only edges with a correlation coefficient above 0.09 are shown (Figure 4 ). An unsupervised weighted Louvain algorithm [39] for community detection was used for this purpose, and the vertices were colored the same if they belonged to the same community, which revealed a meaningful structure in which orange denoted systemic, green denoted local, and yellow denoted gastric communities of symptoms. We compared our results with two available datasets of AEs in response to the Sputnik V vaccine. The first one was collected in Moscow. The second one was collected in Argentine. Mild AEs in 1,029 patients older than 60 years in phase III of the clinical trial [40] were compared with 690 self-reports by Telegram users older than 60 years. Because there were inconsistencies in various definitions of AEs, a simplified classification was provided, and only headache and diarrhea comprised similar symptoms (at least sensu lato). All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 22, 2021. ; https://doi.org/10.1101/2021.06.14.21258875 doi: medRxiv preprint we summed over asthenia and malaise. To obtain nausea, we summed over nausea and dyspepsia. For erythema, we chose contact dermatitis. In all systemic reactions, Telegram users reported AEs significantly more often than measured in the clinical trial (Table 4 ). In contrast, diarrhea was less likely to be reported than measured in the clinical trial. We categorized gastric as the frequency of logical function nausea OR diarrhea. We categorized site irritation as the frequency of logical function pruritus OR erythema OR swelling. We categorized fever_pain as the frequency of logical function fever AND (pain OR headache). We categorized fatigue_pain as the frequency of logical function fatigue AND (pain OR headache). We categorized only_fever as the frequency of logical function fever AND˜(pain OR headache OR fatigue);˜denotes logical negation. The comparison showed that the statistics, despite the significant differences shown in Table 5 , were similar in magnitude and highly correlated (r = 0.94). The comparison of the Telegram reports (a selected sample with at least one AE constructed by multi-label classification) with the Argentinian registry (multi-class classification [41] ) was conducted by the mapping described above. The results of the comparison must be interpreted with caution. 12 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 22, 2021. ; https://doi.org/10.1101/2021.06.14.21258875 doi: medRxiv preprint Regarding vaccines registered by the EMA and FDA, lists of the frequency of the most common adverse events are accessible; however, they vary across regulatory bodies. Thus, we chose a subset of symptoms for frequencies that were reasonably comparable (pain, headache, fatigue, fever, chills, and nausea). We built a distance (Euclidean) matrix of AEs based on clinical trial registries (EMA [38, [42] [43] [44] , FDA [29] [30] [31] ) and from the Telegram group. From the FDA dataset, for two-dose vaccines, the dose with higher reactogenicity was selected. In clinical trials, pain is usually considered pain at the injection site. Fever was the sum of pyrexia and fever in the EMA database. EMA used the injection site tenderness/irritation category. However, regarding redness/erythema, the FDA classified swelling and pruritus separately. Thus, erythema was not included. Sputnik V is a vector vaccine, as are AstraZeneca and Johnson & Johnson. The results showed that Telegram Sputnik V AEs were clustered with other vector vaccines, which was possibly due to similar safety profiles ( Figure 5 ). It is important to note that the Telegram users also submitted reports without any AEs at all. Thus, our surveillance system included a sentinel property of samples in contrast to VAERS (North America), ARR (European Union), and the Argentinian registry [41] , which gather reports only if there is any AE to be reported. 13 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 22, 2021. ; Mild, non-severe AEs have usually been ignored by medical communities because they are common to all vaccines. Antivax movements have emphasized severe AEs, which have been widely discussed in social media [45] in the wider context of vaccine safety [46, 47] . In the discourse on COVID-19 vaccines, the main issues were that they were developed quickly, and they could compromise safety. Those issues included the fear that vaccines would alter human DNA, cause allergic reactions to vaccine ingredients, result in sudden deaths due to frailty syndrome, or cause infertility [48, 49] . Wide anti-COVID immunization programs promulgated a discourse in which risk (e.g., the discomfort of common, but mild AEs as well as rare, but serious AEs) and benefits (e.g., efficacy in protecting from the disease) were described as "tradeoffs" of being vaccinated. Mild AEs have become an important issue for many people; moreover, they have the economic component of the potential need for sick leave. This discourse led to the formation of a public Telegram group, where users were asked to report AEs. 14 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 22, 2021. ; In the present study, we demonstrated that in the first phase of the vaccination roll-out the AE reports were correlated (correlation r = .7) with vaccination volume (Figure 1) . However, the Telegram users tended to lose interest after a few months. It is possible that because of the prioritization of vaccine delivery, which began with public and military servants, scientists, teachers, and medical staff, these "early adopters" were more likely to post on social media and be actively involved in reporting AEs. Subsequently, users in the general population were vaccinated, and they were less involved in reporting on the Telegram platform ( Figure 1 ). The results of the present study showed that the number of reported AEs decreased linearly according to age (β = .05 AE per year; Figure 3 ). This result was dependent on biology, which was confirmed in previous clinical trials [30, 31, 42] and post-marketing observations [37] of other anti-COVID-19 vaccines. Telegram users older than 60 years reported significantly more systemic AEs compared with their peers in clinical trials, who tested negative for having or having recovered from COVID-19 [40] (Table 4) . On one hand, it is possible that people previously infected with COVID-19 were more likely to report AEs after receiving other vaccines [37] . On the other hand, self-reporting bias could be an important factor in explaining the difference between the "Moscow's" clinical trial and the Telegram reports. The safety profile of the Sputnik V vaccine includes mild AEs, which is more similar to vector vaccines than to mRNA anti-COVID vaccines, which was quantified by the Euclidean distance between AE frequencies ( Figure 5 ). Sputnik V safety profile also showed a high fever to fatigue ratio (Table 6 ) and a stronger reaction to the first dose than to the second one (Table 3) . Women reported more AEs than men did (1.2-fold, P < 0.001 Mann-Whitney U test). This phenomenon is well recorded in other anti-COVID-19 vaccine registries [37, 50] , which could indicate sex-dependent vaccine reactivity. However, this result needs to be understood with caution. The CDC has warned that gender bias in reporting could be more important than possible biological mechanisms [37] . The likelihood of disclosing personal information (even anonymously) is known to vary, such as according to gender [51] , social class [52] and so forth. A potential reason is that women are more likely to be interested in health, write about health on the Internet, and disclose their information [51] . On Telegram, self-reports are most likely to underestimate gastric symptoms (e.g., diarrhea 0.6%). These symptoms could be a taboo effect [53] , such as a response to public speaking anxiety. 15 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 22, 2021. ; Alternatively, it could be easily ignored because of its high prevalence, or it could be eliminated using an over-the-counter medicine such as loperamide [54, 55] . Insomnia was detected so often that it suggests an epidemiological link with the vaccine, which needs further investigation. Local AEs, such as injection site irritation, have rarely been reported. Underlying conditions of erythema/redness, which is usually one of the most common AEs in responses to all injected substances, including vaccines, are probably overlooked due to low subjective discomfort and lack of physical investigation by a doctor. The findings showed that their actual prevalence was probably underreported. Our study has several limitations. First, we analyzed participatory/community-based surveillance among Russian Telegram users. Therefore, the results may be specific to the Russian population in a given stage of the pandemic and therefore should not be extrapolated to other contexts. Second, Telegram users may overlook less troublesome side effects, and the social context could influence decisions on taking part in discussions and being selective in reporting AEs [51, 52] . For example, local or gastric AEs could be underreported. Third, the classifications developed in this study should not be strictly applied in other contexts. For example, pain at the injection site and pain in other parts of the body were not distinguished. Finally, because our infodemiology study focused on community research initiatives (independent and nonprofit projects, with already known strengths and weaknesses from the history of medicine [56] ), our observations cannot replace real-world studies [57] [58] [59] . The symptoms reported by social media users only partially reflect their prevalence in the real world [60] . Therefore, the frequencies of symptoms should not be interpreted without considering the contexts and proportions of other symptoms (i.e., fever to fatigue ratio), the phase of the epidemic, and the vaccination roll-out (i.e., the number of doses administered daily and the population that is vaccinated as willingness to report AEs satisfy typical product life-cycle temporal characteristics [60] ). According to clinical trials [40] and official registries [41] , only partial information could be retrieved on Sputnik V. Previously, multiple researchers have raised concerns about the safety of the Sputnik V vaccine [6, 57, 58] . Our study aimed to increase transparency regarding the safety and efficacy of Sputnik V [59] like the US-based early warning objective of actively surveilling COVID-19 vaccine safety through v-safe [61] . 16 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 22, 2021. ; https://doi.org/10.1101/2021.06.14.21258875 doi: medRxiv preprint In the present study, we showed that community-based surveillance via social media (so-called civil "clinical trials") can provide meaningful information that could be useful, and this phenomenon should be carefully investigated. The frequencies of AEs extracted from Telegram samples in which at least one AE was reported were in line with the official Argentinian data on passive surveillance (r = .94 correlation). Because the authenticity and credibility [14] of the reports were not assessed in our study, it is likely that incorrect information was included in the data. Nonetheless, social media monitoring is promising for gathering information on common mild AEs of not only vaccines but also other pharmaceutical products [62] . In the future, it would be of interest to explore the extraction of AEs from a dataset using named entity recognition methods that consider noisy data [11] and to apply nested NER methods [63] . MK received remuneration for performing vaccinations against COVID-19 in primary care. The vaccinations did not involve Sputnik V. Государственный реестр лекарственных средств, Гам-КОВИД-Вак Комбинированная векторная вакцина для профилактики коронавирусной инфекции Safety and immunogenicity of an rAd26 and rAd5 vector-based heterologous prime-boost COVID-19 vaccine in two formulations: two open, non-randomised phase 1/2 studies from Russia Safety and efficacy of an rAd26 and rAd5 vector-based heterologous prime-boost COVID-19 vaccine: an interim analysis of a randomised controlled phase 3 trial in Russia Most Approve of National Response to COVID-19 in 14 Advanced Economies Russia seeks to make Sputnik V in Italy as overseas demand surges Covid-19: What do we know about Sputnik V and other Russian vaccines? Methods for Social Media Monitoring Related to Vaccination: Systematic Scoping Review. JMIR public health and surveillance Syndromic surveillance using web data: a systematic review. Innovation in Health Informatics Searching for Truth: Internet Search Patterns as a Method of Investigating Online Responses to a Russian Illicit Drug Policy Debate A neural network algorithm for extracting pharmacological information from russian-language internet reviews on drugs Medication and adverse event extraction from noisy text Predicting the performance of online consumer reviews: A sentiment mining approach to big data analytics Prospective evaluation of adverse event recognition systems in Twitter: Results from the Web-RADR project Authenticity and credibility aware detection of adverse drug events from social media Identifying adverse effects of HIV drug treatment and associated sentiments using Twitter. JMIR public health and surveillance Complementing the US Food and Drug Administration Adverse Event Reporting System With Adverse Drug Reaction Reporting From Social Media: Comparative Analysis. JMIR Public Health and Surveillance Combining Social Media and FDA Adverse Event Reporting System to Detect Adverse Drug Reactions Frequent discussion of insomnia and weight gain with glucocorticoid therapy: an analysis of Twitter posts Content analysis of Twitter in relation to biological treatments for chronic inflammatory arthropathies: an exploratory study Methods to compare adverse events in Twitter to FAERS, drug information databases, and systematic reviews: proof of concept with adalimumab Evaluation of Facebook and Twitter monitoring to detect safety signals for medical products: an analysis of recent FDA safety alerts. Drug safety Commercial online social network data and statin side-effect surveillance: a pilot observational study of aggregate mentions on facebook. Drug safety Exploring Social Media Network Landscape of Post-Soviet Space Leading social media platforms in Russia as of 3rd quarter of 2020, by penetration rate Adverse event detection by integrating twitter data and VAERS Jarynowski A. Sputnik V Adverse Events risk calculator Systemic Reactions, Adverse Events, and Serious Adverse Events: Janssen COVID-19 Vaccine Systemic Reactions, Adverse Events, and Serious Adverse Events: Moderna COVID-19 Vaccine Systemic Reactions, Adverse Events, and Serious Adverse Events: Pfizer-BioNTech COVID-19 Vaccine Pre-training of Deep Bidirectional Transformers for Language Understanding No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity A systematic analysis of performance measures for classification tasks. Information Processing Management A global database of COVID-19 vaccinations Vaccine side-effects and SARS-CoV-2 infection after vaccination in users of the COVID Symptom Study app in the UK: a prospective observational study. The Lancet Infectious Diseases Fast unfolding of communities in large networks Safety and efficacy of the Russian COVID-19 vaccine: more information needed-Authors' reply. The Lancet Campaña Nacional de Vacunación contra la COVID-19. 10°Informe de vigilancia de seguridad en vacunas 2 de abril code 2021 COVID-19 mRNA Vaccine Moderna-SUMMARY OF PRODUCT CHARACTERISTICS COVID-19 mRNA Vaccine Comirnaty-SUMMARY OF PRODUCT CHARACTERISTICS Vaccine misinformation on social media-topic based content and sentiment analysis of Polish vaccine-deniers' comments on Facebook Anti-vaccine activists, Web 2.0, and the postmodern paradigm-An overview of tactics and tropes used online by the anti-vaccination movement We don't vaccinate") Internet forum. Przegląd epidemiologiczny COVID-19 Vaccine Hesitancy in Canada: Content Analysis of Tweets Using the Theoretical Domains Framework Covid-19 vaccine hesitancy on English-language Twitter. Profesional de la informacion (EPI) First month of COVID-19 vaccine safety monitoring-United States Self-reported symptoms and healthcare seeking in the general population-exploring "The Symptom Iceberg Combining social media affordances for organising collective action co nieuniknione: ekskrementy i defekacja Analysis of Answers to Queries among Anonymous Users with Gastroenterological Problems on an Internet Forum Psychophysiological and self-reported reactivity associated with social anxiety and public speaking fear symptoms: Effects of fear versus distress Impure science: AIDS, activism, and the politics of knowledge Covid-19: Sputnik vaccine rockets, thanks to Lancet boost Covid-19: What do we know about Sputnik V and other Russian vaccines?. bmj opinion Data discrepancies and substandard reporting of interim data of Sputnik V phase 3 trial-Authors' reply Diffusion of innovations Preliminary Findings of mRNA Covid-19 Vaccine Safety in Pregnant Persons Digital drug safety surveillance: monitoring pharmaceutical products in twitter Neural Architectures for Nested NER through Linearization The authors acknowledge the creators and users of the Telegram group "Sputnik_results".