key: cord-0855983-1e46dcz4 authors: Katsuki, Masahito; Matsuo, Mitsuhiro title: Relationship Between Medical Questionnaire and Influenza Rapid Test Positivity: Subjective Pretest Probability, “I Think I Have Influenza,” Contributes to the Positivity Rate date: 2021-07-28 journal: Cureus DOI: 10.7759/cureus.16679 sha: 59898211147849a2c4b2d1e1fd375cb7c93ca4d8 doc_id: 855983 cord_uid: 1e46dcz4 Introduction Rapid influenza diagnostic tests (RIDTs) are considered essential for determining when to start influenza treatment using anti-influenza drugs, but their accuracy is about 70%. Under the COVID-19 pandemic, we hope to refrain from performing unnecessary RIDTs considering droplet infection of COVID-19 and influenza. We re-examined the medical questionnaire’s importance and its relationship to the positivity of RIDTs. Then we built a positivity prediction model for RIDTs using automated artificial intelligence (AI). Methods We retrospectively investigated 96 patients who underwent RIDTs at the outpatient department from December 2019 to March 2020. We used a questionnaire sheet with 24 items before conducting RIDTs. The factors associated with the positivity of RIDTs were statistically analyzed. We then used an automated AI framework to produce the positivity prediction model using the 24 items, sex, and age, with five-fold cross-validation. Results Of the 47 women and 49 men (median age was 39 years), 56 patients were RIDT positive with influenza A. The AI-based model using 26 variables had an area under the curve (AUC) of 0.980. The stronger variables are subjective pretest probability, which is a numerically described score ranging from 0% to 100% of “I think I have influenza,” cough, past hours after the onset, muscle pain, and maximum body temperature in order. Conclusion We easily built the RIDT positivity prediction model using automated AI. Its AUC was satisfactory, and it suggested the importance of a detailed medical interview. Both the univariate analysis and AI-based model suggested that subjective pretest probability, “I think I have influenza,” might be useful. In Japan, rapid influenza diagnostic tests (RIDTs) are considered essential tools for determining when to start the treatment for influenza using anti-influenza drugs, in addition to behavioral measures like hand washing and using masks. This is because the Japanese people want to know the results of RIDTs rather than a qualitative clinical diagnosis by a doctor, although the diagnostic accuracy of RIDTs is around 70-80% [1] . Also, in Japan, patients need the certification of the RIDT results, regardless that they are positive or negative, to go to work or be on sick leave when they have colds during the influenza pandemic season. Therefore, Japanese clinicians routinely use RIDTs as a complementary exam with influenza-like illnesses, and most positive patients are treated with anti-influenza drugs [2] . Doctors also tend to depend on RIDTs because there are few specific physical findings to influenza [3] . However, the treatment for influenza is sometimes initiated based on clinical diagnosis despite negative RIDTs. Therefore, unnecessary RIDTs are being conducted, putting pressure on medical costs. Furthermore, under the COVID-19 pandemic, RIDTs should be performed for patients with a high pretest probability of influenza considering droplet infection of both COVID-19 and influenza. To reduce unnecessary RIDTs and prepare for the telemedicine era under the COVID-19 pandemic [4] , we herein re-examined the medical interview's importance and its relationship to the positivity of RIDTs. Then we preliminarily built a prediction model for the positive rate of RIDTs using detail medical interview results and one of the automated artificial intelligence (AI) frameworks, Prediction One (Sony Network Communications Inc., Tokyo, Japan) [5] [6] [7] [8] [9] [10] [11] . We retrospectively retrieved data from medical records of all the consecutive 96 patients who underwent RIDTs at the outpatient department from December 2019 to March 2020. We used a questionnaire sheet before conducting RIDTs, and doctors performed RIDTs using ESPLINE Influenza A and B-N® (Fujirebio, Tokyo, Japan). We collected the sample at the epipharynx using a swab through the nasal cavity. The questionnaire sheet consisted of the following 24 items: the time duration from the onset, maximum body temperature, vaccination status, pretest probability numerically described as a subjective probability of having influenza ranging from 0% to 100%, and contact with a sick person. We also asked about the presence of detailed symptoms on a three-point scale: heat sensation, tiredness, chills, cold sweat, muscle pain, muscle pain, joint pain, headache, sneezing, shoulder ache, runny nose, nasal congestion, sore throat, cough, sputum, stomach ache, nausea, diarrhea, lower back pain, and feeling "kowai," a local word describing vague physical ailments ( Table 1) . Results were derived as median (interquartile range). The difference between patients with RIDT positive and those negative was examined using univariate analysis with logistic regression. A two-tailed p < 0.05 was considered statistically significant. We calculated p-values and odds ratios (OR) using EZR, which is a graphical user interface for R (The R Foundation for Statistical Computing, Vienna, Austria) [12] , and SPSS Statistics software version 21.0.0. (IBM, Armonk, New York, USA). For numerical variables, the adjusted odds ratios were computed by the exponentiation of the coefficient calculated by the logistic regression. We did not perform multivariate analysis due to the small sample size and many variables, so we built a prediction model using the automated AI framework, Prediction One, instead [5] . We used Prediction One to build the prediction model. We used all 96 patients' data with 26 variables, including the 24 items described above plus sex and age. The objective variable is positive or negative in the RIDTs. Prediction One read the 96 patients' data and automatically divided them into five-fold crossvalidation datasets. Prediction One automatically adjusted and optimized the variables that are easy to be processed statistically and mathematically and select appropriate algorithms with ensemble learning. The missing values were automatically compensated, and Prediction One built the best prediction model by an artificial neural network with internal five-fold cross-validation. The details of functions are trade secrets and could not be provided. The area under the curve (AUC) of the model and strong variables with the weights in the artificial neural network were automatically calculated. The model's AUC, accuracy, precision, recall, and F-value were used to evaluate the prediction model built by AI. Our hospital's research ethics committee approved this study (ethical approval number 2019-20). We gained written informed consent for this study from all of the patients, the legally authorized representative of the patients, or next of kin. All methods were carried out under relevant guidelines and regulations (Declaration of Helsinki). The clinical characteristics of the 96 patients (47 women and 49 men) are summarized in Table 2 The stronger variables and their weights of the model are listed in Table 3 . Pretest probability, presence of cough, past hours after the onset, presence of muscle pain, maximum body temperature, presence of sputum, feeling "kowai," age, presence of joint pain, and cold sweat are important factors, in that order. AI: artificial intelligence, RIDTs: rapid influenza diagnostic tests, *the score numerically described as a subjective probability of having influenza ranging from 0% to 100%, † a local word describing vague physical ailments We administered a carefully designed medical questionnaire to the patients with influenza-like symptoms. We found that muscle pain, cough, and pretest probability, a numerically described subjective score ranging from 0% to 100% of "I think I have influenza," were related to the positive rate in RIDTs in the univariate analyses. Also, we built result prediction model for RIDTs using Prediction One from the 96-patient dataset with the detailed medical questionnaire. Our results offer two suggestions: 1) pretest probability was related to the positivity of RIDTs, and 2) this is the first report on an AI-based prediction model using a detailed medical questionnaire for the positivity rate of RIDTs. Clinical diagnosis is essential before laboratory tests to increase pretest probability. It is ideal to perform RIDTs on people with suspected influenza based on clinical diagnosis. However, in Japan, RIDTs are uniformly performed on patients with influenza-like symptoms during influenza epidemics without a careful medical interview. This is because there are so many outpatients with cold symptoms during winter that we cannot provide usual medical services with careful medical interviews. Also, patients need "certification" based on the RIDTs' results to be allowed to go to work or be on sick leave, so too many patients come to the outpatient department. Therefore, too many meaningless RIDTs are performed in Japan, and it is desired to reduce them to save the limited medical resources. In a guideline on the diagnosis and management of influenza [13] , it was suggested selected individuals should be tested for influenza if the results are anticipated to influence management decisions or a public health response. Therefore, acute febrile respiratory illnesses in patients who are not at high risk for influenza complications and who do not require hospital admission can sometimes be diagnosed as influenza with a high likelihood by clinical criteria alone [14] . In a retrospective analysis of signs and symptoms in 3,744 ambulatory patients with an influenza-like illness who participated in phase II and III trials of neuraminidase inhibitors during outbreaks, the best predictor was the combination of fever and cough within 48 hours of the development of symptoms, which had a positive predictive value of 79% for influenza [1] . Boivin et al.'s study on children suggested that cough and fever were the only clinical factors associated with positive RT-PCR (reverse transcription polymerase chain reaction) results [15] . Call et al.'s study revealed that a combination of fever, cough, and acute onset is related to influenza with a 2.0 positive likelihood ratio [16] . These studies have shown that clinical symptoms help diagnose influenza. However, there are no reports on the usefulness of the subjective pretest probability of having influenza ranging from 0% to 100%. Our report is the first report on its utility. Previous studies on influenza diagnosis using AI remain few. Xia et al. reported an AI-based classifier distinguishing influenza from COVID-19 using chest X-ray images and clinical features with an AUC of 0.9 [17] . Choo et al. reported the usefulness of the patient-generated health data obtained from a mobile health application to develop an AI-based screening tool for influenza [18] . However, Japan has lagged behind other countries in introducing such mobile health applications based on smartphones or smartwatches [19] . Matsui et al. developed a non-contact influenza screening system within 30 s using the newly developed testers for vital signs [20] . Dagdanpurev et al. then developed an AI-based screening model for influenza using those patients' vital signs [21] . Lopez et al. studied AI-based influenza detection systems using natural language processing technology and concluded that a free-text record would help detect influenza [22] . Although AIbased influenza screening systems are being developed, they are still few. Our study is similar to the previous reports on the attitudes to reduce unnecessary RIDTs and predict RIDTs' test results with AI technology. We suggested a possibility that a detailed medical interview could predict RIDTs' results depending on AI technology without contact with patients. In the future, an automatically diagnosing system using Japanese natural language processing [8, 23] for the free text in the medical questionnaire could be invented as a developmental form of our model. Conventional time-and cost-consuming statistical analysis need the arbitrary selection of variables based on previous studies, and multivariate analysis needs 10-15 folds number of samples against the variables [24] . Therefore, there is a risk that variables that might be important may not be included in the statistical analysis. Even the multivariate analysis cannot be performed in a small hospital with a small dataset like ours. However, automated AI like Prediction One develops beneficial models with less effort or time using the small dataset, without time-consuming variable optimization or arbitrarily choosing variables. Also, the number of variables used in the framework is not limited, and AI sometimes finds interesting variables that have not been taken into account in the previously reported statistical models as important [25] . We then review these benefits in our study. Conventionally, we could have used only nine variables for the statistical analysis due to the small sample size of the dataset (n = 96). However, we could use 26 variables for building the prediction model by Prediction One, and also build a good prediction model from the small dataset. Furthermore, we could not perform multivariate analysis due to the small sample size, but Prediction One suggested important variables as if AI considered confounders. Besides, the time needed for creating each model was less than 40 seconds. Finally, the models achieved high accuracy with an AUC of 0.980. We could not perform validation due to the small sample size, but our study suggested the possibility of making the RIDT positivity prediction model only from the detailed medical questionnaire. This technology would help us refrain from performing unnecessary RIDTs considering the medical costs and the COVID-19 pandemic. It also spurs the developing telemedicine era. First, the sample size was so small that we did not perform validation. We only detected important variables using univariate analysis and automated AI. The model's performance is unknown for validation. Second, it is uncertain that the prediction model derived from our own data can be applied to other institutions and those using other RIDTs. We easily and quickly built the RIDT positivity prediction model using Prediction One. Its AUC is satisfactory, and it suggested to us the importance of a detailed medical interview. Especially, subjective pretest probability "I think I have influenza" would be useful. Although validation process is needed, our AIbased prediction model suggested the potential that influenza could be diagnosed with a detailed medical interview and that our preliminary results might be used for further AI-related research like telemedicine and automated diagnosis using natural language processing. Human subjects: Consent was obtained or waived by all participants in this study. Itoigawa General Hospital Ethics Committee issued approval 2019-20. Our hospital's research ethics committee approved this study (ethical approval number 2019-20). We gained written informed consent for this study from all of the patients, the legally authorized representative of the patients, or next of kin. All methods were carried out under relevant guidelines and regulations (Declaration of Helsinki). Animal subjects: All authors have confirmed that this study did not involve animal subjects or tissue. Conflicts of interest: In compliance with the ICMJE uniform disclosure form, all authors declare the following: Payment/services info: All authors have declared that no financial support was received from any organization for the submitted work. Financial relationships: All authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. Other relationships: All authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work. Clinical signs and symptoms predicting influenza infection Influenza vaccine effectiveness against influenza A in children based on the results of various rapid influenza tests in the 2018/19 season Posterior pharyngeal wall follicles as a diagnostic marker of influenza during physical examination: considering their meaning and value (Japanese) Online consultation for influenza screening (Japanese) Sony Network Communications: Prediction One Preliminary development of a prediction model for daily stroke occurrences based on meteorological and calendar information using deep learning framework Easily created prediction model using deep learning software (Prediction One, Sony Network Communications Inc.) for subarachnoid hemorrhage outcomes from small dataset at admission Preliminary development of a deep learning-based automated primary headache diagnosis model using Japanese natural language processing of medical questionnaire Prediction model of deep learning for ambulance transports in Kesennuma city by meteorological data Postsurgical functional outcome prediction model using deep learning framework (Prediction One, Sony Network Communications Inc.) for hypertensive intracerebral hemorrhage Easily created prediction model using automated artificial intelligence framework (Prediction One Investigation of the freely available easy-to-use software 'EZR' for medical statistics Clinical practice guidelines by the Infectious Diseases Society of America: 2018 update on diagnosis, treatment, chemoprophylaxis, and institutional outbreak management of seasonal influenzaa Diagnosis of seasonal influenza in adults Predicting influenza infections during epidemics with use of a clinical case definition Does this patient have influenza? A rapid screening classifier for diagnosing COVID-19 Influenza screening via deep learning using a combination of epidemiological and patient-generated health data: development and validation study Labour and Welfare. Appropriate use of the 'home electrocardiograph program' and 'home heart rate monitor program A novel screening method for influenza patients using a newly developed non-contact screening system A novel machine-learningbased infection screening system via 2013-2017 seasonal influenza patients' vital signs as training datasets Comparison of machine learning classifiers for influenza detection from emergency department free-text reports Predicting inpatient falls using natural language processing of nursing records obtained from japanese electronic medical records: case-control study A simulation study of the number of events per variable in logistic regression analysis Machine learning approach in predicting clinically significant improvements after surgery in patients with cervical ossification of the posterior longitudinal ligament 2021 The authors of this article -MK and MM -have contributed equally.