key: cord-0909976-9euxwiaf
authors: Kim, Kyung Min; Kim, A Ra; Lee, Wonwoo; Jang, Bo Hyun; Heo, Kyoung; Chu, Min Kyung
title: Development and validation of a web-based headache diagnosis questionnaire
date: 2022-04-29
journal: Sci Rep
DOI: 10.1038/s41598-022-11008-y
sha: c3d4313b9301ac3f24129c745846924c315a22c2
doc_id: 909976
cord_uid: 9euxwiaf

Information technology advances may help in conducting epidemiological studies using web-based surveys. Questionnaire-based headache diagnosis should be validated against the doctor’s diagnosis. This study aimed to develop and validate a web-based diagnostic questionnaire for migraine, probable migraine (PM), and tension-type headache (TTH). We constructed a seven-item questionnaire for diagnosing migraine, PM, and TTH. A web-based survey was conducted among adults aged 20–59 years; migraine, PM, and TTH were diagnosed based on the responses. Validation interview was performed via telephone by a neurologist within 1 month after the web-based interview. Finally, 256 participants completed both web-based survey and validation interview. Of them, 121 (47.3%), 65 (25.4%), 61 (23.8%), and 9 (3.5%) were diagnosed with migraine, PM, TTH, and unclassified headache (UH), respectively in the web-based survey, whereas 119 (46.5%), 60 (23.4%), 74 (28.9%), 2 (0.8%), and 1 (0.4%) were diagnosed with migraine, PM, TTH, UH, and primary stabbing headache, respectively in the validation interview. The best agreement was found in migraine (sensitivity: 92.6%; specificity: 94.8%; kappa coefficient: 0.875), followed by TTH (sensitivity: 78.4%; specificity: 98.4%; kappa coefficient: 0.809). PM showed the least agreement (sensitivity: 85.0%; specificity: 92.9%; kappa coefficient: 0.757). In conclusion, our questionnaire is valid in identifying these headache disorders.

The gold standard for diagnosing a headache disorder is via clinical interview by a neurologist using diagnostic criteria 1 . Diagnosis of headache using a self-administered questionnaire is an attractive method for clinical and epidemiological studies as it is inexpensive to conduct and relatively easy to use. Nevertheless, it should be evaluated for its validity by comparing it with the doctor's diagnosis via clinical interview 2 . Several studies have been conducted for validating the diagnostic questionnaires for headache disorders [2] [3] [4] [5] [6] [7] .

With recent advances in information technology, web-based surveys have been introduced and are increasingly used for a variety of purposes. Recently, web-based surveys have been used in epidemiological studies for diagnosing migraine and other headache disorders [8] [9] [10] . They showed similar findings to traditional interview surveys, but differences were also observed 10, 11 . Chronic Migraine Epidemiology and Outcomes (CaMEO) was a web-based survey for migraine while the American Migraine Prevalence and Prevention (AMPP) study collected mailed questionnaires. Both studies sampled from panels representative of the population of the USA. The proportion of individuals who had chronic migraine was similar between the two studies (AMPP, 6.6%; CaMEO, 8.8%) while the response rate was higher in the AMPP (65.8%) study than in the CaMEO (16.5%) study.

The aims of the present study are: (1) To develop a web-based headache diagnostic questionnaire for common primary headache disorders including migraine, probable migraine (PM), and tension-type headache (TTH); (2) to validate a web-based headache diagnostic questionnaire by comparing it with the doctor's diagnosis via telephonic interview.

| (2022) 12:7032 | https://doi.org/10.1038/s41598-022-11008-y www.nature.com/scientificreports/ the web-based survey. Among the 65 participants with PM in the questionnaire-based diagnosis, 37 (56.9%) did not meet the criterion of typical headache characteristics, 19 (29.2%) did not meet the criterion of typical headache duration, and 9 (13.8%) did not meet the criterion of accompanying symptoms. In the validation interview, 119 (46.5%), 60 (23.4%), 74 (2.9%), 2 (0.8%), and 1 (0.4%) were diagnosed with migraine, PM, TTH, UH, and primary stabbing headache, respectively. Among 60 participants with PM in doctor's diagnosis via telephone interview, 33 (55.0%) did not meet the criterion of typical headache characteristics, 19 (31.7%) did not meet the criterion of typical headache duration, and 8 (13.3%) did not meet the criterion of accompanying symptoms. One participant had a primary stabbing headache. Demographic and clinical characteristics of the participants are summarized in Table 1 . The age and sex distribution of the participants were not significantly different from those in the general population of Korea (Supplementary Table S1 ).

Validity of headache diagnoses. The sensitivity, specificity, and positive and negative predictive values of migraine, PM, and TTH diagnosis and agreement with kappa statistics between the web-based survey and validation interview of all 256 participants are shown in Table 2 . Kappa coefficients for migraine and TTH showed very good agreements whereas that for PM showed a good agreement (Table 2) . We also conducted analyses among 101 participants with low frequency headaches (< 1 attack/month) ( Table 3) . Kappa coefficient for migraine revealed a very good agreement and those for PM and TTH demonstrated a good agreement. A subgroup analysis among 32 participants with high frequency headaches (> 6 attacks/month) showed a very good agreement for migraine and good agreements for PM and TTH (Table 4 ).

We developed a seven-item web-based diagnostic questionnaire for migraine, PM, and TTH based on the ICHD-3 criteria 12 . We found that our questionnaire showed very good or good agreement in the diagnosis of migraine, PM, and TTH when compared to a neurologist's diagnoses.

Various studies have been conducted for validating a questionnaire in the diagnosis of migraine 2-7 by comparing questionnaire-based diagnoses and clinical interview-based diagnoses. Most studies have shown that the kappa coefficients range from 0.43 to 0.77, which suggests a moderate to good agreement. However, our study showed a kappa coefficient of 0.875, which suggests a very good agreement. The high rate of agreement in diagnosing migraine was also confirmed in the subgroup analyses. Therefore, we could confirm that we had developed a highly valid questionnaire for diagnosing migraines.

Web-based surveys have several aspects that differ from those of the traditional interview surveys. They need shorter time, less cost to conduct, and make the access to unique populations and data quality control easy. www.nature.com/scientificreports/ Moreover, real-time access is available and interviewers are not required for these surveys 13 . Its disadvantages include survey fraud, limited sampling and respondent availability, difficulty in generating samples from an online community, and less experience of the individuals in undergoing a web-based survey 14 . For the optimal use of the web-based survey questionnaire, its validity should be properly evaluated 2 . Nevertheless, the validation of a web-based migraine or headache diagnostic questionnaire has rarely been reported. An American study, Penn www.nature.com/scientificreports/ Online Evaluation of Migraine, evaluated a web-based questionnaire for diagnosing migraines, which included 59 individuals with migraine and 31 headache-free control, and found a sensitivity of 59% and a specificity of 84% in diagnosing migraines 15 . The present study included sufficient sample size and showed that our questionnaire had a very good or good agreement in the diagnosis of migraine, PM and TTH. Our study found that the diagnostic accuracy of PM was lower compared to that of migraine an TTH. Similar results were also found in a computer-based questionnaire study in China 16 . The study revealed that Kappa coefficients of PM and probable TTH were 0.698 and 0.639, respectively when compared to doctor's diagnosis. In contrast, Kappa coefficients of migraine and TTH were 0.961 and 0.921, respectively. Therefore, it should be noted that the diagnostic accuracy of 'probable' headache diagnosis was lower from that of 'strict' headache diagnosis when questionnaire-based headache diagnosis applied.

A better agreement was noted in cases where there was a shorter time interval between the initial survey and the validation interview 7 . A guideline for headache epidemiology study recommended that re-interviewing should not be conducted more than once in a month 1 . All validation interviews were performed within 1 month after the web-based survey in the present study. This short interval between the initial survey and validation interview was made possible by the web-based survey, and it might have contributed to the high agreement observed in this study.

Web-based headache diagnosis could be useful in several ways in addition to clinical and epidemiological studies. Approximately half of individuals with migraine did not know their migraine diagnosis 17, 18 . Web-based diagnostic questionnaire can be a way to increase awareness of migraine diagnosis. Clinical decision systems have shown effectiveness for improving clinical practice 19, 20 . Thus, a clinical decision system based on the headache diagnosis using web-based questionnaire would be a good choice in the management of headache 16, 21, 22 . Also, web-based questionnaire could promptly assess changes in headache characteristics and in treatment strategies during public health emergency situation like coronavirus disease 2019 pandemic 23, 24 . Our web-based questionnaire showed a high diagnostic accuracy for migraine, PM and TTH, and will provide a good opportunity not only in the diagnosis of headache but also in the management of headache disorders.

The present study has several limitations. First, the distribution of headache diagnosis was different from that of the general population. Migraine prevalence in the present study was 46.4%, which was higher than that in epidemiological studies in Asian countries, which ranges between 5.1 and 9.0%. A higher prevalence of migraine in this study might be attributed to the recruitment bias in the web-based survey 1, 25 . Individuals with migraine might have more interest in a headache survey than those without migraine and this could have resulted in a higher prevalence of migraine in the present study. Although some validation studies of headache diagnostic questionnaires were conducted in populations with different headache characteristics from the main survey population 2,7 , the validity should be evaluated in the relevant populations for the proper use of the web-based survey questionnaires. Second, we did not assess the demographic features of non-participating individuals. Identifying characteristics of non-participating individuals is an important issue in population-based studies 26 . Nevertheless, we could not obtain demographic characteristics of non-participating individuals owing to the personal information protection act in Korea. Third, we included a sufficient sample size of participants for the evaluation of the diagnostic validity of migraine, PM, and TTH in the present study. Nevertheless, optimal sample size for some subgroup analyses might not have been maintained. Although we found that our questionnaire showed good or very good agreements in the participants with low frequency and high frequency headaches, this point is a limitation of our study. Lastly, the overall cooperating rate in the present study was not high. We recruited participants who had at least one headache attack during the previous 1 year and agreed to participate in the validation interview via a mobile phone. These conditions might have contributed to the low cooperation rate. The lower cooperation rate in a web-based survey than that in a traditional mail survey has already been reported 8, 11 . Nevertheless, the age and sex distribution of our participants were not significantly different from that in the total population of Korea.

The present study also has several strengths. First, we developed and validated a diagnostic questionnaire for PM in combination with migraine and TTH. PM is a common headache disorder, and its symptoms and disabilities are similar or somewhat milder than those of migraine 26 . Some epidemiological study evaluated its prevalence and clinical characteristics using a questionnaire. Nevertheless, diagnostic validity of the questionnaire has not been specifically evaluated yet. This is the first validation study for the diagnosis of PM. Second, we included enough participants to evaluate the sensitivity and specificity of the web-based questionnaire in diagnosing migraine, PM, and TTH. By having sufficient sample size in the present study, we can confirm that the diagnostic validity of our web-based questionnaire has been properly evaluated.

In conclusion, we developed a web-based questionnaire for diagnosing migraine, PM, and TTH based on the ICHD-3 criteria. By comparing the web-based survey diagnosis and the diagnosis by a neurologist, we found that our web-based questionnaire showed a very good agreement for migraine and TTH and a good agreement for PM. The findings of this study suggest that our web-based headache diagnostic questionnaire is a valid tool for diagnosing migraine, PM, and TTH.

We developed a questionnaire for the headache diagnosis based on the ICHD-3 criteria of migraine, PM, and TTH. Since both criteria of migraine without (code 1.1) and with aura (code 2.1) were composed of five criteria including the total number of attacks, duration of attacks, typical headache characteristics, accompanying symptoms, and a poor accountability by another headache diagnosis questionnaire, we constructed seven questions for the duration of attack, typical headache characteristics, and accompanying symptoms 12 (Table 5 and Supplementary Table S2 ). A question for headache frequency was also included. We did not consider criteria A and E for the total number of attacks and a poor accountability by another headache diagnosis questionnaire in migraine without aura and infrequent episodic TTH. Most individuals with headaches could not recall the exact number of headache attacks. Moreover, adult individuals with at least one attack of headache during the previous 1 year may have had experienced more than 10 headache attacks during their lifetime. Approximately 98% of individuals with headaches had primary headache disorders, and it is very difficult to diagnose headaches other than common primary or secondary headache disorders in epidemiological studies 12,27 . Participants. We recruited participants for the present study via a mobile phone text invitation among the survey panels of Hankook Research. An invitation mobile phone text was sent with the title 'a survey on headaches' . The invitation text messaging was not stratified by sex or age. The inclusion criteria were: (1) aged 20-59 years (2) having at least one headache attack during the previous 1 year, (3) able to participate in the webbased headache survey, and (4) agreed to participate in an additional telephonic interview by a doctor. Exclusion criteria were as follows: (1) aged < 20 or > 59, (2) did not experience any headache attack during the previous 1 year, and (3) did not agree to participate in an additional telephonic interview.

Web-based survey. After accepting the participation request, internet link for the web-based survey was sent to all the participants, and they were asked to complete the survey. The web-based survey was conducted from May 1 to May 10, 2021.

A semi-structured telephonic validation interview was performed by a neurologist with a special interest in headache medicine (MKC) using a case report form (Supplementary Table S2 ). All participants were asked about (1) the occurrence of headache during the previous 1 year, (2) duration of headache attacks, (3) intensity of headache, (4) typical headache characteristics including unilateral pain, pulsating quality, aggravation by routine physical activity, and (5) accompanying symptoms of nausea, vomiting, photophobia, Table 5 . Web-based questionnaire for the diagnosis of headache.

These questions about your "most bothersome headache" during the previous 1 year Q1. On an average, how long did these headaches last? (1) Headache did not disturb usual daily activities (mild)

(2) Headache often disturbed usual daily activities, but I could perform more than half of my daily activities ( Sample size calculation. Sample size calculation was based on the prevalence, confidence interval, and the estimated sensitivity and specificity of the questionnaire for diagnosing migraine, PM, and TTH. We set the estimated prevalence of migraine, PM, and TTH as 15%, 20%, and 60%, respectively among participants' headaches based on the previous epidemiological studies [28] [29] [30] . To achieve a maximal error of 10%, the sample size for 90% sensitivity and 95% specificity for migraine, PM, and TTH was calculated as 230, 172, and 57, respectively 31 . Therefore, we set the target sample size as 230 in the present study.

Identification of the duration, intensity, characteristics, and accompanying symptoms of headache. We identified the duration, intensity, characteristics, and accompanying symptoms of headache based on the responses to the questionnaire (Table 5 and Supplementary Table S2 ). The duration and intensity of headache were determined based on the response to Q1 and Q3, respectively. Unilateral pain was considered by positive responses to (1), (2), or (5) of Q4. Pulsating quality was recognized by positive responses to (1) or (3) of Q5. Aggravation by routine physical activity was identified by a positive response to Q7. Nausea, vomiting, photophobia, and phonophobia were assigned when a positive response was recorded to (2) Since it is very difficult to accurately identify the presence of aura using questionnaire, we did not evaluate the presence of aura 32 . Therefore, participants with migraine in the present study included both migraine with and without aura. Similarly, PM included both PM with (code 1.5.2) and without (code 1.5.1) aura. If a participant's headache fulfilled all but one criteria of migraine, the participant was diagnosed as having PM. Diagnoses of TTH was based on criteria B through D for infrequent TTH (code 2.1) in ICHD-3. Participants who met all these criteria were considered to have TTH. We did not apply the frequency criterion (criterion A) in the diagnosis of TTH. Thus, the TTH evaluated in this study included infrequent TTH (code 2.1), frequent TTH (code 2.2), and chronic TTH (code 2.3). When a participant was diagnosed with TTH and PM simultaneously, a diagnosis of TTH was assigned 12 . If a participant's headache did not fulfill the criteria for migraine, PM, and TTH, the participant was diagnosed with an UH.

During the validation interview, all headache diagnostic criteria for migraine (code 1.1, criteria A-E), PM (code 1.5, criteria A-C), and TTH (code 2.1, criteria A-E) were applied. Headache diagnoses other than migraine, PM, TTH, and UH were also allowed.

Ethical considerations. This study was approved by the institutional review board/ethics committee of Severance Hospital, Yonsei University (IRB No. 2021-0538-001). Written informed consent was obtained from each participant. All procedures involving human participants were performed in accordance with the ethical standards of the institutional and/or national research committee as well as the tenets of the 1964 Declaration of Helsinki and its later amendments, or comparable ethical standards.

Statistical analyses. Sensitivity, specificity, positive and negative predictive values, accuracy, and kappa coefficients with 95% confidence interval were calculated for different headache diagnoses based on the responses in the web-based questionnaire using a neurologist's headache diagnoses as a gold standard. A kappa coefficient of ≤ 0.20 was considered as poor, between 0.21 and 0.40 as fair, between 0.41 and 0.60 as moderate, between 0.61 and 0.80 as good, and between 0.81 and 1.00 as a very good agreement 33 

The methodology of population surveys of headache prevalence, burden and cost: Principles and recommendations from the Global Campaign against Headache

Head-HUNT: Validity and reliability of a headache questionnaire in a large population-based study in Norway

Migraine in adolescents: Validation of a screening questionnaire

Prevalence and sex-ratio of the subtypes of migraine

Prevalence of migraine and other headaches in Hong Kong

Questionnaire versus clinical interview in the diagnosis of headache

The HUNT4 study: The validity of questionnaire-based diagnoses

The impact of chronic migraine: The Chronic Migraine Epidemiology and Outcomes (CaMEO) Study methods and baseline results

The International Burden of Migraine Study (IBMS): Study design, methodology, and baseline cohort characteristics

The applicability of web-based solutions in headache epidemiology research

A Comparison of the Chronic Migraine Epidemiology and Outcomes (CaMEO) Study and American Migraine Prevalence and Prevention (AMPP) Study: Demographics and headacherelated disability

Headache Classification Committee of the International Headache Society (IHS) The International Classification of Headache Disorders

Advantages and disadvantages of online surveys

Researching internet-based populations: Advantages and disadvantages of online survey research, online questionnaire authoring software packages, and web survey services

A web-based, branching logic questionnaire for the automated classification of migraine

Validation of a guideline-based decision support system for the diagnosis of primary headache disorders based on ICHD-3 beta

Patterns of diagnosis and acute and preventive treatment for migraine in the United States: Results from the American Migraine Prevalence and Prevention study

Self-awareness of migraine: Interpreting the labels that headache sufferers apply to their headaches

Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: A systematic review

Why don't physicians follow clinical practice guidelines? A framework for improvement

Validation of an algorithm for automated classification of migraine and tensiontype headache attacks in an electronic headache diary

A survey on probable and improbable decisions about headache treatment

Hospital-based headache care during the Covid-19 pandemic in Denmark and Norway

Primary headaches during the COVID-19 lockdown in Germany: Analysis of data from 2325 patients using an electronic headache diary

My Migraine Voice survey: A global study of disease burden among individuals with migraine for whom preventive treatments have failed

Reasons for non-participation and ways to enhance participation in health examination surveys-the Health

Comprehensive application of the international classification of headache disorders third edition, beta version

Prevalence of migraine in Taipei, Taiwan: A population-based survey

Anxiety and depression in probable migraine: A population-based study

Anxiety and depression in tension-type headache: A population-based study

Sample size calculations in studies of test accuracy

Impact of migraine in the United States: Data from the National Health Interview Survey

Statistical methods for assessing observer variability in clinical measures

Anonymized data relevant to this study will be shared by request with qualified investigator pending appropriate Institutional Review Board approvals.

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIT) (2019R1F1A1053841).

The online version contains supplementary material available at https:// doi. org/ 10. 1038/ s41598-022-11008-y.Correspondence and requests for materials should be addressed to M.K.C.Reprints and permissions information is available at www.nature.com/reprints.Publisher's note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.