key: cord-0714774-wdr39wwv
authors: Taylor-Phillips, S.; Berhane, S.; Sitch, A.; Freeman, K.; Price, M.; Davenport, C.; Geppert, J.; Harris, I.; Osokogu, O.; Skrybant, M.; Deeks, J. J.
title: Information given by websites selling home self-sampling COVID-19 tests: An analysis of accuracy and completeness
date: 2020-08-19
journal: nan
DOI: 10.1101/2020.08.18.20177360
sha: dc3b702f4c52b9fae64620145dd869cc9a517555
doc_id: 714774
cord_uid: wdr39wwv

Objectives: To assess the accuracy and completeness of information provided by websites selling home self-sampling and testing kits for COVID-19. Design: Cross-sectional observational study. Setting: All websites (n=27) selling direct to user home self-sampling and testing for COVID-19 (41 tests) in the UK (39 tests) and US (2 tests) identified by a website search on 23rd May 2020. Main outcome measures: Thirteen predefined basic information items to communicate to a user, including who should be tested, when and how testing should be done, test accuracy, and interpretation of results. Results: Many websites did not provide the name or manufacturer of the test (32/41; 78%), when to use the test (10/41; 24%), test accuracy (12/41; 29%), and how to interpret results (21/41; 51%). Sensitivity and specificity were the most commonly reported test accuracy measures (either reported for 27/41 (66%) tests); we could only link these figures to manufacturers documents or publications for four (10%) tests. Predictive values, most relevant to users, were rarely reported (five [12%] tests reported positive predictive values). For molecular virus tests, 9/23 (39%) websites explained that test positives should self-isolate, and 8/23 (35%) explained that test negatives may still have the disease. For antibody tests, 12/18 (67%) websites explained that testing positive does not necessarily infer immunity from future infection. Seven (39%) websites selling antibody tests claimed the test had a CE mark, when they were for a different intended use (venous blood rather than finger-prick samples). Conclusions: At the point of online purchase of home self-sampling COVID-19 tests, users in the UK are provided with incomplete, and in some cases misleading information on test accuracy, intended use and test interpretation. Best practice guidance for communication about tests to the public should be developed and enforced for online sales of COVID-19 tests.

Results: Many websites did not provide the name or manufacturer of the test (32/41; 78%), when to use the test (10/41; 24%), test accuracy (12/41; 29%) , and how to interpret results (21/41; 51%). Sensitivity and specificity were the most commonly reported test accuracy measures (either reported for 27/41 (66%) tests); we could only link these figures to manufacturers' documents or publications for four (10%) tests. Predictive values, most relevant to users, were rarely reported (five [12%] tests reported positive predictive values). For molecular virus tests, 9/23 (39%) websites explained that test positives should self-isolate, and 8/23 (35%) explained that test negatives may still have the disease. For antibody tests, 12/18 (67%) websites explained that testing positive does not necessarily infer immunity from future infection. Seven (39%) websites selling antibody tests claimed the test had a CE mark, when they were for a different intended use (venous blood rather than finger-prick samples).

Conclusions: At the point of online purchase of home self-sampling COVID-19 tests, users in the UK are provided with incomplete, and in some cases misleading information on test accuracy, intended use and test interpretation. Best practice guidance for communication about tests to the public should be developed and enforced for online sales of COVID-19 tests.

• We believe this is the first research on accuracy of information provided by websites selling tests for COVID-19, where users may put themselves or others at increased risk of transmission if results are misinterpreted. • We duplicated processes of searching and data extraction to minimise bias • Using pre-specified criteria, we found evidence that websites selling home self-sampling COVID-19 tests provided incomplete and inaccurate information on test accuracy and interpretation of test results at the point of purchase. • We developed basic guidance on what should be communicated when selling tests, including the type of test; situations when the test should be used; the time when the test should be done and details of how it should be done; the name of the test and details from clinical accuracy studies; evidence of compliance with regulatory approvals; explanation of test results using accessible and relevant metrics such as predictive values; and guidance to the interpretation and actions based on results. • We only included websites from the UK and US, so whilst the principles of what should be communicated apply to all countries, the results about data completeness are not generalisable beyond the UK and US.

The 2019 novel coronavirus (COVID- 19) pandemic has resulted in national population measures such as restricted movement ('lockdown'), and mass testing programmes. Testing is regarded as critical to manage the pandemic -the two main test types available being molecular virus tests (to detect current infection) and antibody tests (to detect previous infection). The World Health Organization (WHO) recommends polymerase chain reaction (PCR)-based molecular virus testing of symptomatic individuals to detect current COVID-19 infection, 1 to enable identification and isolation of confirmed cases, and tracing of those exposed for further testing. However, due to the sensitivity for a single PCR test being as low as 70%, 2 the WHO states that even two consecutive negative PCR tests do not rule out infection with COVID-19. 1 Antibody tests are not recommended for individual use by the WHO, because we do not yet understand whether presence of antibodies infers immunity from future infection. Their sensitivity has been estimated at around 80-90%, thus there is also a risk of false negatives. 3 Timing of testing is critical for both tests: molecular tests are thought to be most accurate when used within five days of the onset of symptoms, 4 antibody tests are most accurate two or more weeks after onset of symptoms. 3 There are now multiple websites selling both molecular virus tests and antibody tests outside of national testing programmes. To ensure appropriate use, interpretation and actions following testing, it is necessary for tests to be sold with clear communication about who should use each test, when and how samples should be taken, and the implications of positive and negative results. Previous research investigating direct to user sales of genetic testing found that the information provided was incomplete, particularly the implications of test results and limitations of testing, and was not always in an accessible and understandable format. [5] [6] [7] Direct to user sale of tests are regulated by the Medicines and Healthcare products Regulatory Agency (MHRA) in the UK and the Food and Drug Administration (FDA) in the US. Europe 8 and the USA 9 operate a risk based regulation for in vitro diagnostic devices (IVDs) which depends on the intended use of the test and indications for use. IVDs for home testing fall into higher risk categories reflecting the fact they are initiated, performed and interpreted without professional guidance and require evidence that lay users correctly use the test and understand test results. Lay user studies are required as the basis for the instructions for use (IFU) document for the IVD. 10 Home sampling tests are different from home testing as they receive approval based on home collected specimens with the test analysis being undertaken by professionals. At the time of writing, there were no COVID-19 antibody tests with a CE mark for either home sampling or home testing 11 13 where the appropriateness of the test purchase is assessed by a professional either pre-purchase or following a purchase request.

We analysed the information given to individuals considering purchasing a molecular virus or antibody COVID-19 test online for home self-sampling. We chose to review tests for sale in both the UK and US to cover two different regulatory systems with contrasting health services. We recorded information regarding who should be tested and when, claims about test accuracy, and information 4 about how to interpret results. As the MHRA instigated a withdrawal of sales of antibody tests based on finger-prick blood samples on May 29 th 2020 where tests require venous blood samples, 11 we also evaluated how test vendors have responded.

Our research question was how complete, accurate and informative is the information that online websites selling home self-sampling and testing for COVID-19 provide to the public?

The search was designed to identify a representative sample of websites and online advertisements which would be seen by an individual searching for a non-specific COVID-19 test. We aimed to identify websites selling home self-sampling and testing for COVID-19 using molecular virus and/or antibody tests directly to users. Two researchers performed the searches independently on the same day (23 rd 

We included websites selling molecular virus and/or antibody tests for COVID-19 direct to users in either the US or UK. We included point-of-care and laboratory-based tests, with the proviso that the sample was taken at home by the individual themselves. We excluded tests with assisted sampling (e.g. drive-through testing), or where part of the testing process before purchase included video, telephone or in-person contact with a medical professional (as we could not objectively assess the information content of such interactions). We included websites selling tests both via direct purchase and insurance funding, but excluded local or national government websites providing tests (including Public Health England [PHE] and the UK National Health Service [NHS]), and websites providing tests as part of a research study. We included all eligible tests, including where a single website sold multiple eligible tests. We excluded websites with a minimum order of more than a single test, as these targeted suppliers rather than individual users.

We extracted information about the test manufacturer and type of test; when testing was recommended; claims made about test accuracy; the advice given about changing behaviour in light of test results; accreditation; and the test cost. We assessed the information provided against a predefined list of items which we would expect to be communicated to a person considering purchasing a test for COVID-19, detailed in Table 1 .

We extracted claims made about regulatory approval of the tests, in particular CE-IVD approval in the UK and FDA approval in the US, and where possible compared claims to the actual approval status for the test. We also extracted claims made about approval from non-regulatory bodies such as PHE and the NHS.

Website contents were extracted between 23 rd and 28 th May 2020. One researcher extracted data from each website onto a predefined data extraction form, and downloaded the website as an image file. A second researcher checked each extraction using the pdf copy to exclude temporal changes.

. CC-BY-NC-ND 4.0 International license It is made available under a perpetuity.

is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted August 19, 2020. . https://doi.org/10.1101/2020.08.18.20177360 doi: medRxiv preprint

A public contributor (MS), with both experience of being involved in research and leading public involvement in research, provided input into this project. MS has an interest in communicating scientific information to lay audiences. The rapid timeframes in which the research was conducted limited the scope for more comprehensive public involvement. MS contributed to discussions, paper drafts and is included as a co-author.

Ethics approval was not required for this review of publicly available documents.

For the UK our Google searches retrieved 550 results, and for the US they retrieved 430 results. After the first round of sifting by 2 reviewers 46 potentially eligible websites were identified. Of these 19 websites were later excluded, 13 of which only sold in quantities greater than one or to laboratories/hospitals/workplaces, 5 who incorporated contact with a health professional before the sale, and one which was withdrawn from sale between the search and extraction. We identified 23 molecular virus testing services 14-36 and 18 antibody testing services 14 35 36 ). One website 40 did not appear in the main search, but was mentioned in many UK news articles, so was included in the cohort. Only two websites using home sampling were identified in the US, the first and second to be approved by the FDA for this use. 35 30 32 34 40 four suppliers of a range of direct to consumer testing online, 18 22 31 37 three laboratories, 23 33 35 two online sexual health specialists 19 27 and one supplier of beauty treatments. 28 All 23 molecular virus tests were laboratory-based tests with home sampling. Of the 18 antibody tests, 17 were laboratorybased tests with home self-sampling, and one was a point-of-care test. 38 The test manufacturer was identifiable for 9/41 (22%) tests, further details are provided in Supplementary Figure 1 .

The mean cost of molecular virus testing was £168 (range £65 to £279) in the UK and $135 (range $119 to $150) in the US. The mean cost of antibody tests was £87 (range £55 to £130) in the UK.

The proportion of websites which met each of the criteria for clear communication (outlined in Table  1 ) is shown in Figure 1 , and examples of reporting are given in Box 1.

All 27 websites stated whether the 41 tests for sale were molecular virus tests or antibody tests, of which 40/41 described the test clearly. Guidance on timing of taking the molecular virus tests and the antibody tests was provided by 15/23 (65%) 15 

Of the 41 tests for sale, the websites reported a measure of test accuracy (sensitivity, specificity, positive or negative predictive value) for 27 (66%) tests: 16/18 (89%) for antibody tests 14 16 32 and eight molecular tests 15 16 18 19 23 27 29 32 ) only reported test performance using unclear terms such as 'accuracy' or 'reliability', for example "This test has a 99.9% accuracy" 19 and "This test offers 99.9% reliability". 29 Tests with unclear performance values may be referring to analytical performance, such as "Our test is sensitive to fewer than 100 copies of the target viral is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted August 19, 2020. . https://doi.org/10.1101/2020.08.18.20177360 doi: medRxiv preprint RNA, making it a highly accurate test." 32 For two (5%) molecular tests, no text or values referring to accuracy were reported on the websites. 24 31 Sensitivity and specificity were the most commonly reported accuracy measures, provided for 27/41 (66%) 14 15 39 40 tests, respectively. Sensitivity estimates ranged from 95% to 100% for antibody tests (n=16) 14 15 17 20-22 25 26 34-36 Five of the 41 tests (13%; two antibody tests 28 31 and three molecular 14 20 33 tests) provided an estimate or statements of sensitivity and/or specificity under conditions of perfect use rather than pragmatic use, for example "If there are any coronavirus on your swab it will definitely find it". 33 No websites directly referred to positive predictive values (PPV), but they were indirectly reported for 5/41 (12%) tests. 25 25 21 20 40 Two antibody 25 40 and three molecular tests 25 21 20 made a statement about the lack of false positives (implying a PPV of 100%), for example "if it shows a positive result, it can only be for COVID-19". 25 No cross-reactivity (meaning the test would not identify other viruses, only COVID-19 virus) was referred to by websites for 13/41 (32%) tests (five antibody 16 25 33 34 40 and eight molecular tests 17 20-22 25 26 28 34 ). Negative predictive value (NPV) was not referred to by any websites; however, statements implying that the NPV was less than 100% were given for 4/41 (10%) available tests (two antibody 25 31 and two molecular tests 20 35 ), for example "The test can sometimes show a negative result even if you are infected SARS-CoV-2, the virus that causes COVID-19". 35 The number of samples used to generate accuracy data were given for 5/41 (12%) tests; two antibody tests 31 33 and three molecular tests. 22 35 36 Accuracy data were linked to a journal publication for only 1/41 (2%) test. 33 Information on interpreting both positive and negative molecular virus test results was presented for 4/23 (17%) websites. 20 33-35 Twelve of the 18 (67%) websites selling antibody tests informed potential customers prior to purchase that a positive antibody test may not infer immunity from future infection 14 16 Where tests could be identified, we checked accuracy claims against data from published papers, pre-prints (based on information obtained from searches from ongoing Cochrane reviews for these tests) and manufacturer's data in the Instructions for Use (IFU) sheet for each test (Table 2) . Four websites reported clinical performance data for the Abbott IgG antibody test: two 31 33 quoted the performance figures from the IFU 41 , for the other two 26 29 no exact match with available studies could be made. Of the four molecular tests, no performance data were available for the Randox test 23 (including in the IFU 42 ), no direct match of clinical performance results could be made for the website selling the Primerdesign genesig PCR assay 22 (where the IFU only reported data from contrived samples 43 ), whereas the data reported by US websites 35 36 selling the LabCorp and Rutgers PCR tests, respectively, matched data from the manufacturers' IFUs. 44 45 

Across the 25 UK websites, there were 17 antibody tests for home sampling, 14 is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted August 19, 2020. . https://doi.org/10.1101/2020.08.18.20177360 doi: medRxiv preprint tests were based on approved test use by health professionals using venous rather than finger-prick samples:

"All of our home test kits are CE-marked. This is one of the two IgG tests approved by the Government for UK use." 15 One website 38 claimed it had regulatory approval for its home testing antibody test:

"Our test has been accepted by Medicines and Healthcare products regulatory Agency (MHRA), which means that it can be applied across the EU including UK. We confirm our product can meet the requirement of In vitro Diagnostic Medical Devices Directive (98/79EC) and standards complying with CE Declaration of Conformity." 38 Five of 21 (24%) UK websites selling molecular virus tests for home sampling included a statement that they had regulatory approval 15 21 25 26 33 and 6 (29%) websites 17 20 28 32-34 claimed approval from a policy making body for this intended use. The manufacturer or name of the molecular tests for which websites were claiming regulatory approval or endorsement could not be identified. Only for two websites selling molecular tests 22 23 , the test manufacturer could be identified, neither of which made any claims about regulation or endorsement. One of these tests 23 is mentioned by the UK government as part of its COVID-19 testing strategy 46 and the other 22 was one of the tests which was independently evaluated by PHE. 47 Both USA websites selling molecular viral tests 35 36 have approval from the FDA for home sampling during the COVID-19 pandemic. These websites included information about the eligibility checks that purchasers would need to undergo either prior to purchase or prior to test processing.

We reviewed the 18 UK websites selling home COVID-19 antibody tests 14-16 18 19 21 25 26 28 29 31-34 37-40 on 11 th /12 th June 2020 after the MHRA had instructed sales of these tests to cease because of the lack of approval for the tests using finger-pick samples. 11 We found two websites 32 38 that appeared to still be selling finger-prick tests, four 14 21 28 31 had switched to providing a venous blood sampling service, two 18 33 required the purchaser to find their own phlebotomist to draw a blood sample to send, six 15 16 19 25 34 40 simply stated that tests were out of stock and were unavailable, whilst four 26 29 37 39 reported the MHRA guidance and indicated that they had suspended sales (Table 3) .

We identified 27 websites selling COVID-19 tests direct to the public, 25 in the UK but only two in the US, which may be explained by the FDA stipulations requiring clinician involvement in the testing process. We observed that many websites failed to provide complete information on the name and manufacturer of the test (no information for 32/41 tests), when to use the test (no information for 10/41 tests), the accuracy of the test (no information for 12/41 tests), and how to interpret results (no information for 21/41 tests), which will hinder the public making informed choices about testing, using tests correctly and understanding what test results mean. Without adequate and correct information the public may purchase the wrong or a poor test, or use the test in the wrong way or at the wrong point in time. These errors or application will increase their chances of getting an erroneous test result. Even when used properly, few websites assisted users in interpreting test results and understanding their inherent uncertainty.

This rapid evaluation was designed to provide timely results in the context of a fast-moving global pandemic. The search was not designed to be exhaustive, rather to represent what a person typing "coronavirus test" or similar into a Google search would have retrieved. Using different phrases such is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted August 19, 2020. . https://doi.org/10.1101/2020.08.18.20177360 doi: medRxiv preprint as "coronavirus antibody test" would have identified additional websites, but there is no reason to suspect that they would be different from those summarised here. The timing of the search and data extraction will have affected results. Data extraction was shortly before the UK MHRA clarified that antibody tests were not approved for finger-prick samples, only for venous samples. The search only identified two US websites selling tests with home sample collection, but at the point of going to press eight tests are now approved on the FDA website. 48 The criteria that we used to assess completeness of communication (detailed in Table 1 ) were defined a priori, but due to time constraints a formal process for developing these was not followed. However, all key elements of the search, selection and data extraction processes were undertaken independently by two researchers, reducing the possibility of errors. We only assessed information provided prior to purchase, as complete information should be given at this stage to inform the purchasing decision. However, further information would have been given after purchase, for example within the instructions for use, which was beyond the scope of this paper.

The issues we have identified are examples of poor and misleading practice, and some merit further investigation by the MHRA and Advertising Standards Authority. At the time of going to press two antibody tests remained on sale. The communication of test accuracy appears to contravene advertising standards in the UK. The five websites that reported PPV of 100% contrary to the wider evidence base, and all websites making accuracy claims which is not linked to supporting evidence appear to contravene section 12.1 of the ASA code, 49 which states that objective claims must be backed by evidence. Further, websites provided specificity and sensitivity, or general claims of 'accuracy' rather than positive and negative predictive values explained in lay terms, and the ASA have previously ruled against this practice as misleading in the case of non-invasive prenatal testing for trisomies. 50 Finally, the lack of complete information on the implications of positive and negative test results does not appear to be covered by any UK regulation, perhaps because the ASA 12.2.1 49 prohibit diagnosis by post or email, and so this information is intended to be provided by contact with a healthcare professional. Whilst such contact with a health professional is happening in the US it does not appear to be in the UK. Regulation of product labelling provides a means to oversee information communicated for self-testing products bought in person, but there is currently no equivalent for online testing services in the UK. This gap in regulation could be solved by expanding the responsibility of the MHRA to include communication by 'distributors' at the point of online purchase, working collaboratively with the Advertising Standards Agency. There was a large variation in price of testing in the UK, and in many cases these differences do not appear to be justified by differences in the service provided. Greater regulation and standardisation of website claims may reduce this price differential by making comparisons between websites easier, and removing unsubstantiated claims.

It is important that all test users are given adequate and appropriate information to help them make safe and informed choices. We identified five key communication issues with websites selling direct to consumer home-sampling COVID-19 tests. All five of these issues may be improved by developing a basic framework of what information should be provided, and standard ways to present such information. This would also facilitate comparison between websites.

It is essential that companies selling tests identify the type of test, and the situations in which it is appropriate to order such a test. Whilst websites were clear whether they were selling molecular or antibody tests, they also need to indicate the situations when it is appropriate to order a molecular is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted August 19, 2020. . https://doi.org/10.1101/2020.08.18.20177360 doi: medRxiv preprint "swab" test or an antibody "blood" test in order to select the correct one. The two US websites utilised questionnaires recording symptoms and exposure which were reviewed by clinicians prior to tests being despatched which provides a more rigorous check on whether the test request is sensible.

Both molecular and antibody tests need to be used at different time points in the disease course. The sensitivity of both types of tests will fall if used at the wrong time point (sensitivity of 31% for antibody tests in the first week since onset of symptoms 3 ), substantially increasing the risks of infection or antibody response being undetected. Recommended time points when samples should be taken were absent for 10/39 UK tests (26%). Some timing statements were misleading, suggesting using the test at time points which are known to be too early or too late. Some websites stated dates based on time since exposure, others since symptom onset which is median 5 days after exposure. Both are required to be able to advise both asymptomatic patients and patients with unknown exposure when they should order and use the tests.

Websites must also describe the full testing process and clearly indicate what is required of users to complete testing. For example, two antibody websites currently indicate that purchasers will need to identify individuals qualified to take venous blood samples, which is impractical for most people.

3) The test name, evidence of its accuracy, and evidence of its regulatory approval for the purpose to which it is put.

The majority of tests were for sale by third parties, ranging from healthcare providers to beauty treatment specialists. In most cases (32/41; 78%) it was not possible to identify the test being used or the manufacturer. This does not allow the individual to know the product that they are buying, and precludes the opportunity for the user to verify its regulatory status and the claims being made.

Information on test accuracy was absent or uninterpretable for 12/41 (29%) tests. Numerical accuracy claims could only be matched to published evidence for 4/41 (10%) of tests. In these instances, figures most closely matched those from the manufacturers' Instructions for Use leaflets, which tended to report the highest observed values of sensitivity and specificity, and were based on studies more akin to analytic validity than clinical validity evaluations. Accuracy measures from analytic validity studies should not be assumed to give a good representation of test accuracy when applied in practice to the public. A wide range of terms were used, several of which did not have a clear meaning. It appeared that test accuracy data is not available at all for some tests (Randox) 42 , or only based on contrived samples (Primerdesign) 43 and not on real patients. For molecular COVID-19 tests, no clinical performance data were available that were based on self-sampled swab tests. Withholding the fact that there is no patient based evidence of the accuracy of these tests from the public is unacceptable. It is important that the reported accuracy is based on all reviewed evidence and not selected results, and clearly explains how applicable the evidence is to the public.

Naming tests is essential to be able to check their regulatory status. Seven of 18 (39%) UK websites selling antibody tests inappropriately claimed CE marking, when the CE marking was for a different intended use (venous rather than finger-prick blood samples). Antibody tests are not approved for home use in the US, and none were found in our search. The UK regulator acted after we had reviewed UK websites, clarifying that antibody tests which are approved for the use with venous samples should not be sold for the use with finger-prick samples. However, 2/18 remain available for online sale at the point of going to press (accessed 11 th to 12 th June 2020). The molecular tests we could identify are approved for home sampling, however, the name and manufacturer was not identifiable for most websites. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted August 19, 2020. . https://doi.org/10.1101/2020.08.18.20177360 doi: medRxiv preprint

Research concerning the communication of test accuracy evidence is limited and is largely restricted to self-selected, professional and postgraduate student groups. 51 Communication of test accuracy evidence is complex for several reasons. Research has highlighted the importance of communicating the potential consequences of positive and negative test results (use of predictive values) and the importance of contextualising estimates of accuracy with reference to a healthcare setting (for example hospital in patient, hospital outpatient, community). 52 53 Presenting test accuracy as frequencies rather than as probabilities improves understanding.

To interpret results, test users need to know how to interpret positive and negative test results (predictive values), not the proportion of cases detected (sensitivity) and non-cases correctly diagnosed (specificity). Positive predictive value was only reported for 5/41 (12%) tests, and in all five they claimed it was 100% which is inconsistent with the broader evidence base. Negative predictive value was not reported at all.

Most websites gave insufficient information regarding interpreting test results. Only 8/23 (35%) websites explained that a negative molecular virus test does not rule out COVID-19, and only 12/18 (67%) explained that a positive antibody test does not necessarily infer immunity from future COVID-19 infection or transmission.

Misunderstanding of the implications of test results could mean that individuals put themselves or others at risk of infection in the mistaken belief that they do not have COVID-19, or that they are immune to COVID-19. This last category probably has the greatest potential for harm. Clear communication about the meaning of test results as detailed above should be linked to evidencebased guidance about behaviour modification in light of test results. We found widespread evidence of websites failing to provide such evidence-based guidance, and some cases of websites actively suggesting unsafe behaviour.

At the point of online purchase of home self-sampling COVID-19 tests, users in the UK are provided with incomplete, and in some cases misleading information on test application, accuracy and interpretation. Many websites omit trustworthy guidance on the timing of tests, the interpretation of positive and negative test results, and the implications of results. Best practice guidance for communication about tests to the public should be developed and the role of the regulator in enforcing complete and accurate information should be reviewed. This should be underpinned by robust collaborative qualitative research exploring how members of the public interpret information and measures of accuracy, thus informing how it can be provided in a way that is clear, complete and accessible . CC-BY-NC-ND 4.0 International license It is made available under a perpetuity.

is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted August 19, 2020. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted August 19, 2020. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted August 19, 2020. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint Linking information on the low negative predictive value of the PCR test to recommendations to continue self-isolation may strengthen the message. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted August 19, 2020. . https://doi.org/10.1101/2020.08.18.20177360 doi: medRxiv preprint This is the most important accuracy information for a person considering buying a test. For an individual whose molecular virus test result is positive, the positive predictive value gives them the probability that they currently have COVID-19. For an individual whose molecular virus test is negative, the negative predictive value is the probability that they do not currently have COVID-19.

For an individual whose antibody test is positive, the positive predictive value is the probability that they have COVID-19 antibodies. For an individual whose antibody test is negative the negative predictive value is the probability that they do not have COVID-19 antibodies. These metrics are dependent on disease prevalence as well as sensitivity and specificity, but can reasonably be calculated with informed estimates of prevalence. 8. Does the website give a link or reference to a journal article of test accuracy?

Indicating the source of these data would help substantiate the claims, and allow interested people to find out more. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted August 19, 2020. Same rationale as item 9 above, but here we assessed whether the websites gave correct information (in addition to avoiding misinformation). 13. Antibody test -does the website explain that we do not know whether a positive test infers immunity, and/or that you shouldn't put yourself at more risk of exposure if you test positive?

Same rationale as item 10 above, but here we assessed whether the websites gave correct information (in addition to avoiding misinformation).

. CC-BY-NC-ND 4.0 International license It is made available under a perpetuity.

is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted August 19, 2020. . https://doi.org/10.1101/2020.08.18.20177360 doi: medRxiv preprint MyHealthcare Clinic 29 The manufacturer of the Antibody test (Abbott Laboratories) reports that an independent clinical performance evaluation of the test performed in the United Kingdom confirmed the following accuracy levels: Sensitivity (ability to identify positive cases) of 99.7%. Specificity (ability to identify negative cases) of 100%.

Bryan et al 2020 55 • Positive samples tested: 125 patients who tested RT-PCR positive for SARS-CoV-2 for which 689 excess serum specimens were available (unclear how many at each time point). The Online Clinic (Online Clinic (UK) Limited) 26 Tests have a sensitivity of 100%. When using a patient-collected sample with one of our home sampling kits, the sensitivity of this test has been shown to reduce slightly to 97.5. Recent studies suggest a specificity of 99.9% and 99.8% respectively. 16 We have a trusted product which is specific to COVID-19 and sensitive. As with all of these kits they undergo regular testing to ensure accuracy and reliability which on the last occasion were 100% accurate for both positive and negative samples. Antibody test is very specific for COVID-19 (some inferior tests can mistake other infections for COVID-19 and wrongly reassure you); it is also very sensitive for the specific IgG antibody. Not currently available "Sorry the item you have selected is not currently available, please choose another option" "Due to the high demand of orders, the Antibodies blood test service is currently not available." Superdrug 40 Not currently available "We have temporarily halted the COVID-19 antibody testing service. If you have any questions please send us a message through your account." Zava 25 Out of stock "This product is temporarily out of stock." Better2Know 19 Out of stock "Currently out of stock"

Website links to guidance from MHRA. Antibody Solutions 28

Out of stock/ Modified test "Please note: These kits are no longer in stock; however, we are offering a full blood sample collection service, either at your home or at one of our partner clinics." Blue Horizon Medicals 18 Modified test "Ordering this test will allow us to send you a vacutainer kit, which allows a healthcare professional to draw a venous blood sample from your arm. You should only order this kit therefore if you have access to a healthcare professional with the appropriate skills. Phlebotomy should NOT be attempted by those who are unskilled." Qured 21 Modified test "A healthcare professional will visit your home to take a venous blood sample."

"The antibody tests currently used by our laboratory are the Abbott test if you opt for venous blood collection by a healthcare professional, or the Siemens test if you opt to collect your blood sample yourself" "These tests are currently validated for venous blood draw only, which is why our service includes an at-home blood draw from a healthcare professional. Home selfcollection of blood using a finger prick kit for antibody testing has been temporarily paused pending evaluation by the Medicines and Healthcare Products Regulatory Agency (MHRA)." CityDoc 14 Modified test "We are able to offer blood collection by the normal practice of intravenous blood sampling at our clinics across the UK and sent to our accredited UK laboratory for testing." Atruchecks Limited 31 Modified test "This option is so you can arrange a home or workplace visit by a phlebotomist to take your blood for you." The Online Clinic (Online Clinic (UK) Limited) 26 Suspended/ Modified test "The self-collect home sampling service is currently suspended but will be back shortly. Please check back later." "The Medicines and Healthcare Regulatory Agency is currently conducting a review of self-collect blood samples for this type of test and the service is unavailable until that review concludes. We now offer a home-sampling service where a phlebotomist attends your home (or other premises) to collect the blood sample from a vein. "

Withdrawn/ Suspended "We have unfortunately had to withdraw the PHE Approved Antibody Home Testing Kits, per the unexpected Government / MHRA ruling on 26 May re private testing. We do not currently have a date for when these Home Tests will be next available to private patients." Medichecks 37 Withdrawn/ Suspended "Currently, the only way to get a private coronavirus antibody test is to buy a venous blood test where you will need to visit a nurse or health professional to have a sample collected from a vein in your arm. All private laboratories and private testing companies have paused self-collect finger-prick testing while the MHRA conducts its review.

. CC-BY-NC-ND 4.0 International license It is made available under a perpetuity.

is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted August 19, 2020. . https://doi.org/10.1101/2020.08.18.20177360 doi: medRxiv preprint However, we are confident that this service will resume shortly once the laboratories have completed their validation studies." Babylon 39 Withdrawn/ Suspended "Important update on COVID-19 Antibody Tests. The MHRA (the government regulator responsible for medicines and medical devices) has asked that all COVID-19 antibody testing from finger-prick blood samples be paused. The MHRA decision has impacted all testing of this type nationwide."

"The lab will not be offering further testing services until the MHRA have provided clearance to do so."

. CC-BY-NC-ND 4.0 International license It is made available under a perpetuity.

is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted August 19, 2020. . https://doi.org/10.1101/2020.08.18.20177360 doi: medRxiv preprint is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted August 19, 2020. . https://doi.org/10.1101/2020.08.18.20177360 doi: medRxiv preprint Figure 2 . Recommendations given by websites on when to take the molecular virus tests and antibody tests. Test accuracy is dependent on correct timing.

Text to add to figure legends: Figure 2 :

No guidance 14 16 22-24 28 32 33 Current symptoms/recent exposure or both 17 18 30 31 1-5 days since exposure 34 1-5 days since symptom onset 15 26 1-5 days since symptom onset or later 20 5 days after exposure 29 At least 5 days after exposure 19 27 Up to 14 days post exposure 21 Established via a questionnaire 25 35 36 No guidance 14 32 At least 14 days post infection 18 At least 14 days post symptom onset 19 33 38 40 At least 14 days post symptoms or exposure 25 26 34 20 days post symptom onset 31 At least 21 days post infection 39 At least 21 days post symptom onset 16 28 29 37 At least 21 days post symptoms or exposure 21 28-35 days (start point not specified) 15 . CC-BY-NC-ND 4.0 International license It is made available under a perpetuity.

is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted August 19, 2020. . https://doi.org/10.1101/2020.08.18.20177360 doi: medRxiv preprint

All authors contributed to the conception of the work and interpretation of the findings. OO and JG performed the Google searches. STP, SB, KF, JG, OO, IMH, and MJP extracted the data. STP, AJS, MJP and CD undertook the analysis and drafted the manuscript. All authors critically revised the manuscript and approved the final version. STP acts as guarantor. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

This paper presents independent research supported by the NIHR Birmingham Biomedical Research 

We have read and understood BMJ policy on declaration of interests and declare the following interests: STP is funded by the NIHR through a career development fellowship; KF is funded by the is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted August 19, 2020. . https://doi.org/10.1101/2020.08.18.20177360 doi: medRxiv preprint from the Contribution to third party material where-ever it may be located; and, vi) licence any third party to do any or all of the above.

Ethical approval not required

No additional data available

The lead authors and manuscript's guarantor affirms that the manuscript is an honest, accurate and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted August 19, 2020. . https://doi.org/10.1101/2020.08.18.20177360 doi: medRxiv preprint

World Health Organization. Laboratory testing for coronavirus disease (COVID-19) in suspected human cases: interim guidance

FALSE-NEGATIVE RESULTS OF INITIAL RT-PCR ASSAYS FOR

Antibody tests for identification of current and past infection with SARS-CoV-2

Ask for an antigen test to check if you have coronavirus now

Internet-Based Direct-to-Consumer Genetic Testing: A Systematic Review

Information Provided to Consumers about Direct-to-Consumer Nutrigenetic Testing

Informed choice in direct-to-consumer genetic testing (DTCGT) websites: a content analysis of benefits, risks, and limitations

REGULATION (EU) 2017/746 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 5 April 2017 on in vitro diagnostic medical devices and repealing Directive 98/79/EC and Commission Decision

Federal Food, Drug and Cosmetic Act (FD&C Act) Chapter V: Drugs and Devices

Guidance for notified bodies on the regulation of IVDs for self-testing

Medicine & Healthcare products Regulatory Agency. Guidance For industry and manufacturers: COVID-19 tests and testing kits

Guidance For patients, the public and professional users: a guide to COVID-19 tests and testing kits

Emergency Use Authorizations

Coronavirus Home Testing Kits

Covid-19 Home Testing Kits

Covid-19 Test kits and Covid-19 Test Antibody kits

COVID-19 Home Coronavirus Test Kits

Covid-19 testing

COVID-19 PCR Home Sample Kit -Currently infected

Covid-19 PCR and Antibody Tests Now Available

COVID-19 Test

COVID-19 Home Testing Kit

Coronavirus antibody and swab testing service

COVID-19

Covid-19 Coronavirus Home Test Kit

Covid-19 Home IgG Sample Collection Test Kit and Lab Test

MyHealthcare Clinic. Coronavirus (COVID-19) Testing

Coronavirus Testing -Home Sample Collection Kit

Covid-19 tests -Private Covid-19 Clinical Testing Service

Covid-19

Coronavirus Antibody & PCR Tests -In Stock Now

Webmed Pharmacy Ltd. covid-19 tests

COVID-19 Test (At-Home Kit

Hims Inc. COVID-19 Saliva Test

Ten minute coronavirus self-testing kit

COVID-19 Antibody Test

SARS-CoV-2 IgG ARCHITECT. Instructions for Use (IFU)

Extended Coronavirus Multiplex Array EV4418 User Manual

Coronavirus (COVID-19) genesig®Real-Time PCR assay -For Emergency Use Authorization Only -Instructions for Use (IFU) Issue

Food & Drug Administration. ACCELERATED EMERGENCY USE AUTHORIZATION (EUA) SUMMARY -SARS-CoV-2 ASSAY (Rutgers Clinical Genomics Laboratory)

Department of Health & Social Care. Coronavirus (COVID-19) -Scaling up our testing programmes. Department of Health and Social Care

Rapid assessment of the Novacyt Primerdesign Coronavirus (COVID-19) genesig® Real-Time PCR assay

Food & Drug Administration

The Committee of Advertising Practice Ltd. (CAP). 12 Medicines, medical devices, health-related products and beauty products -CAP Code

Non-Invasive Prenatal Testing (NIPT) -A look at the ASA's rulings. CAP News

How well do health professionals interpret diagnostic information? A systematic review

Guidance was developed on how to write a plain language summary for diagnostic test accuracy reviews

Understanding test accuracy research: a test consequence graphic

The Incubation Period of Coronavirus Disease 2019 (COVID-19) From Publicly Reported Confirmed Cases: Estimation and Application

Performance Characteristics of the Abbott Architect SARS-CoV-2 IgG Assay and Seroprevalence in

SARS-CoV-2 Antibody responses do not predict COVID-19 disease severity

Evaluation of the Abbott SARS-CoV-2 IgG for the detection of anti-SARS-CoV-2 antibodies. Version of 8

Epitope Diagnostics Inc. EDI™ Novel Coronavirus COVID-19 IgG ELISA Kit (KT-1032). Instructions for Use (IFU)

Comparison of four new commercial serologic assays for determination of SARS-CoV-2 IgG

Rapid assessment of the Randox Extended Coronavirus Multiplex Array assay. V01

Comparison of seven commercial RT-PCR diagnostic kits for COVID-19