Abstract
This prospective observational study, conducted at a community clinic in Japan during the influenza season, from December 2017 to April 2018 aimed to investigate the accuracy of factors used for influenza self-diagnosis. Data were collected from pre-examination checklists issued to patients with suspected influenza and electronic medical records. Receiver operating characteristic (ROC) curve analysis was performed using a rapid influenza diagnostic test as the reference standard, and 2 × 2 contingency tables were analysed at each cut-off point. We analysed data from 290 patients (72.8% males, median age: 38 years, interquartile range: 26–50 years). The area under the ROC curve (AUC) for patients who were aware of other patients presumed to have influenza within close proximity was 0.74 (95% confidence interval (CI): 0.66–0.82). The AUCs for patients with a history of influenza, unvaccinated status, cough, or nasal discharge were 0.68 (95% CI: 0.60–0.75), 0.66 (95% CI: 0.59–0.73), 0.67 (95% CI: 0.59–0.75), and 0.70 (95% CI: 0.62–0.78), respectively. The sensitivity, specificity and positive likelihood ratio at a 90% cut-off point was 19.5% (95% CI: 13.5–26.6%), 94.1% (95% CI: 88.7–97.4%) and 3.31 (95% CI: 1.57–6.98). The sensitivity, specificity and negative likelihood ratio at a 10% cut-off point was 95.5% (95% CI: 90.9–98.2%), 9.6% (95% CI: 5.2–15.8%) and 0.48 (95% CI: 0.20–1.16). After multivariate logistic regression analysis, the AUC increased significantly from 0.77 (95% CI: 0.70–0.83) to 0.81 (95% CI: 0.76–0.86) when self-diagnosis-related information was added to basic clinical information. We identified factors that improve the accuracy and validity of influenza self-diagnosis. Appropriate self-diagnosis could contribute to the containment efforts during influenza epidemics and reduce its social and economic burden.
Similar content being viewed by others
Introduction
Influenza is an acute respiratory disease due to the influenza virus and is a common disease among patients presenting at outpatient clinics (14.6 million people/year in Japan1, 14.5 million people/year in the United States2) resulting in regional or seasonal epidemics (mainly in the winter3) and significant economic costs4,5.
In Japan, a rapid influenza diagnostic test (RIDT) has frequently been used to diagnose influenza. However, the RIDT is reported to have low sensitivity (Sn) (62.3%), with a specificity (Sp) of 98.2% in a meta-analysis6, and not be useful for early diagnosis, especially within 12 h of disease onset7. In many countries, the diagnosis of seasonal influenza is often performed clinically8. Early seasonal influenza is often misdiagnosed; however, if this could be overcome by accurate patient self-diagnosis, there could be potential to limit the spread of infection. Self-diagnosis could also enable viral containment if those diagnosed stay at home while they are infectious. With the development of information and communication technology, self-diagnosis and self-prescription using the information available on the internet have increased9, and interest in the accuracy of self-diagnosis has grown. Therefore, if a more accurate and faster self-diagnosis could be effectively made, it may help prevent the spread of infection, especially in places where medical resources are scarce10.
A retrospective study showed that the self-diagnosis index (extremely high or low probability) was useful for ruling in or ruling out an influenza diagnosis11. However, no studies have examined factors affecting the accuracy of influenza self-diagnosis, and no prospective observational studies have investigated the accuracy of the self-diagnosis index of influenza. Hence, in this study, we aimed to investigate the clinical factors improving the accuracy of the self-diagnosis index of seasonal influenza, to validate self-diagnosis accuracy using a prospective observational study design, and to analyse the synthetic clinical diagnostic value of adding self-diagnosis-related information to basic clinical information.
Results
Patient characteristics
In total, 383 patients met the inclusion criteria, and 290 patients (median age: 38 years; IQR 25%, 75%: 26, 50 years; male, 72.8%) were analysed (see Methods, Fig. 1). The study profile and demographics of the study population are summarised in Fig. 1 and Table 1, respectively.
Area under the receiver operating characteristic (ROC) curve (AUC) of self-diagnosis for each category subgroup
Area under the ROC curve of self-diagnosis for each category subgroup with or without factors expected to affect the self-diagnosis accuracy can be found in Table 2. The AUC for the patient group that was aware of other patients presumed to have influenza located within close proximity to them was 0.74 (95% confidence interval (CI): 0.66–0.82), which was significantly higher than among patient groups who were unaware of other patients presumed to have influenza in close proximity to them. The AUCs of patient groups with a history of influenza infection, unvaccinated status, cough, or nasal discharge were 0.68 (95% CI: 0.60–0.75), 0.66 (95% CI: 0.59–0.73), 0.67 (95% CI: 0.59–0.75), or 0.70 (95% CI: 0.62–0.78), respectively; and these were relatively higher (difference in AUC > 0.05) than the patient groups without these factors. The AUC in the younger patient group (<18 years, n = 28) was 0.72 (95% CI: 0.54–0.91). In the older patient group (≥65 years, n = 11), the AUC was 0.38 (95% CI: 0.04–0.71). The AUC increased to 0.87 (95% CI: 0.75–0.98) in the patient group with all five factors present, apart from the age factor due to the small patient numbers.
ROC curve analysis at multiple cut-off points
We performed ROC curve analysis at multiple cut-off points to verify the accuracy of influenza self-diagnosis. The AUC of influenza self-diagnosis (%) was 0.64 (95% CI: 0.58–0.70). The Sn, Sp, positive likelihood ratio (LR+), and negative likelihood ratio (LR−) were 38.3% (95% CI: 30.6–46.5), 83.8% (95% CI: 76.5–89.6), 2.37 (95% CI: 1.54–3.65), and 0.74 (95% CI: 0.64–0.85), respectively at the optimal cut-off point of 65% (Fig. 2). The LR− at the 10% cut-off point was 0.48 (95% CI: 0.20–1.16) and the LR+ at the 90% cut-off point was 3.31 (95% CI: 1.57–6.98). Other findings concerning Sn, Sp, LR+, and LR− at multiple cut-off points are listed in Table 3. In an additional analysis, 12 negative RIDT result cases were considered to be clinically diagnosed influenza to compensate for the low sensitivity of the RIDT as a reference standard, and the AUC was 0.64 (95% CI: 0.58–0.70), which did not show a significant difference.
The additional clinical diagnostic value of the self-diagnosis
Multivariate logistic regression analysis was performed to analyse the additional clinical diagnostic value of the self-diagnosis (Fig. 3). The AUC increased significantly from 0.77 (95% CI: 0.70–0.83) to 0.81 (95% CI: 0.76–0.86) (P = 0.03) when self-diagnosis-related information (self-diagnosis of influenza (%), awareness of other patients presumed to have influenza located within close proximity to the study patients, past medical history of influenza infection, and influenza vaccination status) was added to basic clinical information (age, sex, cough, headache, nasal discharge, fatigue, joint and muscle pain, and axillary temperature taken at home).
Discussion
We conducted a prospective observational study and identified five important factors (awareness of other patients presumed to have influenza located within close proximity, history of influenza infection, unvaccinated status, cough, and nasal discharge) that increased the accuracy of influenza self-diagnosis. Accessing self-diagnosis-related information in addition to basic clinical information was useful for diagnosing influenza, especially when patients self-reported extremely high or low probability.
Past studies have reported that self-diagnosis, by dichotomous categories with yes and no responses, has low diagnostic accuracy (Sn, 45.7%; Sp, 58.1%)12. However, in a previous retrospective observational study, we showed that the self-diagnosis index was useful because we were able to evaluate the diagnostic accuracy at multiple cut-off points, especially when patients self-reported extremely high or low probability (LR+: 2.75 at 80% cut-off point, 95% CI: 0.75–10.07; LR−: 0.33 at 10% cut-off point, 95% CI: 0.12–0.96; AUC: 0.63, 95% CI: 0.53–0.73)11. In the current prospective observational study, we confirmed the usefulness of the self-diagnosis index (LR+: 3.31 at 90% cut-off point, 95% CI: 1.57–6.98; LR−: 0.48 at 10% cut-off point, 95% CI: 0.20–1.16; AUC: 0.64, 95% CI: 0.58–0.70) (Table 3), which is consistent with our previous results.
Our study had several strengths. For the first time, to our knowledge, we identified five factors that can improve the accuracy of self-diagnosis. Factors such as history of influenza infection, unvaccinated status, cough, or nasal discharge tended to improve the accuracy of self-diagnosis, indicating that the presence of subjective symptoms can render self-diagnosis more accurate. Although there has been no direct research on whether influenza vaccinations reduce the symptoms of influenza, their effectiveness in reducing the incidence of influenza is clear13 and the alleviation of symptoms can negatively affect the accuracy of self-diagnosis. Information regarding a history of influenza infection may have helped patients to distinguish between the common cold and influenza through enabling a comparison between past and current symptoms. Additionally, we showed that patients’ awareness of other patients presumed to have influenza within close proximity significantly improved the accuracy of self-diagnosis. The shape of the graph showing medical visits for influenza at the research clinic was similar to that at sentinel sites in the research area and we consider that our data reflected the epidemic situation in the area (Supplementary Fig. S1). The peak in the number of patients who were aware of patients presumed to have influenza within close proximity to them overlapped with a peak in the number of patients with influenza at the research clinic (Supplementary Fig. S2); therefore, the more patients who visited the clinic, the more self-diagnosis was considered to be clinically useful because the proportion of patients with high diagnostic accuracy of self-diagnosis increased in line with the epidemic. In recent years, several studies have pointed out that local, influenza-related Google search frequency corresponded to local influenza epidemic status14,15. The search frequency for “influenza” in the research area (Aomori Prefecture) during the study period also corresponded to the epidemic situation in the area (Supplementary Fig. S3). As described, there are several methods to confirm a local influenza outbreak, such as direct observation of the area, information derived from a local medical institution, and information provided on the internet. We showed that confirmation of a local influenza outbreak by patients’ direct observations (confirmation at places such as their workplace, school, or home) was the most reliable factor for improving the accuracy of influenza self-diagnosis. Second, this was a prospective observational study and was larger than our previous retrospective study11; therefore, the quality of the collected data was improved, and selection bias was reduced. Moreover, the results of this study increased the reliability of influenza diagnostic accuracy and were consistent with the results and conclusions of previous research. Third, using multivariate logistic regression analysis, this study showed that self-diagnosis-related information was additionally useful for influenza diagnosis in general practice. For diagnostic clinical information to be useful, it is important that it improve overall diagnostic accuracy when added to existing basic clinical information, rather than being used as a form of independent individual information. Finally, this study involved recent baseline data on the self-diagnosis index of influenza. In recent years, many patients have obtained health information on the internet prior to medical visits16,17, which may affect self-diagnosis and treatment behaviour. The cultural background situation in relation to the development of technology is changing rapidly; therefore, additional studies undertaken at appropriate times are necessary.
Our study also had a few limitations. First, the interpretation of our results may differ depending on the age group. In the ≥65 years group, the accuracy of self-diagnosis was extremely poor (Table 2). There were only 11 cases analysed in this group and it was difficult to validate the results statistically. Therefore, the results of this study cannot be extended to elderly populations. Second, we needed to select a more accurate reference standard. In Japan, especially in primary care clinics, it is usually difficult to perform polymerase chain reaction tests or obtain viral cultures, which are highly accurate reference standards for influenza diagnosis. Future studies are needed where more accurate tests can be easily used for influenza diagnosis in primary care settings. Third, patient background factors, such as education levels and different cultures, should be considered. In Japan, the concept of percentages is taught to fifth graders (10- and 11-year olds). Therefore, it may be necessary to change the types of questions used, depending on patients’ age and the education system in each country involved. Fourth, the answers of the pre-examination checklist about influenza were based on participants’ recall and knowledge of encountering someone presumed to have influenza, and we could not check their reliability. However, even if their information affected the accuracy of the self-diagnosis, we believe that it was useful as a daily medical care tool. Finally, a prospective study with the main objective of validating the factors suggested to be effective in this study is required. To use influenza self-diagnosis more effectively, additive disease factors (e.g. strains of influenza), patient factors (e.g. more detailed symptoms, including patients with no symptoms, cognitive function, sources of information, internet, and smartphone use), and environmental factors (e.g. residential areas (such as urban areas), remote areas, or other countries) that were not included in this study should be analysed.
Our results have practical significance in the following two settings in terms of influenza epidemic suppression and social burden reduction. First, in this study, we identified factors for improving self-diagnosis that can be applied to acquiring medical history information in telemedicine as well as in ordinary clinical examinations. In this study, we showed that adding self-diagnosis-related information to basic clinical information significantly improved diagnostic accuracy (Fig. 3), which indicates that the accuracy of influenza diagnosis can be improved without direct face-to-face communication and can be used in telemedicine. In recent years, telemedicine has been reported to be useful not only for the management of chronic diseases18 and for improvement in the quality of emergency care19, but also for initial treatment of epidemic diseases20. Minimising clinical visits of outpatients with mild symptoms involving a low complication risk is important in controlling influenza transmission because visits to a medical institution may result in potential exposure to an influenza virus21. Second, self-diagnosis can contribute to efficient screening by applying different cut-off points (e.g. high Sn at 10% cut-off point, high Sp at 90% cut-off point). In general, self-diagnosis is considered to be useful in situations where medical care is unavailable22, in avoiding burdening the public health system22, or where early diagnosis has a significant effect on disease prognosis10. In situations where infection control is a major social issue (e.g. pandemic influenza), effective action can be taken based on patient risk without assessment at a medical institution.
In conclusion, we identified factors that increased the accuracy of influenza self-diagnosis such as awareness of proximity to others presumed to have influenza. Accessing self-diagnosis-related information in addition to basic clinical information was useful for diagnosing influenza, especially when patients self-reported extremely high or low probability. Appropriate self-diagnosis could contribute to the containment efforts during influenza epidemics and reduce its social and economic burden.
Methods
Study design and ethical considerations
We conducted a prospective observational study at a community clinic in Rokkasyo-mura, Aomori, Japan (Rokkasho Centre for Community and Family Medicine) and collected data on factors affecting the accuracy of influenza self-diagnosis, then verified the self-diagnostic data for accuracy and assessed the additional diagnostic value of self-diagnosis. This study was approved by the Medical Ethics Committee of Hirosaki University (approval number: 2017-1100). All data were fully anonymised at the time of data collection, and written informed consent was obtained from all patients included in the study.
Patients
We invited all patients who were clinically suspected of seasonal influenza (e.g. upper respiratory tract symptoms and fever) to complete a pre-examination checklist during the influenza season, from December 2017 to April 2018. Patients who met all three of the following criteria were included in the study: (1) aged ≥ 12 years, (2) had completed a pre-examination checklist prior to a doctor’s assessment, and (3) had undertaken an RIDT. Patients were excluded from the study if they: (1) did not complete all the items on the pre-examination checklist, or (2) did not provide written consent.
Data collection
Data were collected using pre-examination checklists (Supplementary Figs S4, S5) and electronic medical records. Items included in both data sources reflected not only clinical symptoms but also factors that were presumed to be important for self-diagnosis, as reported in past studies11,23,24,25,26,27, or that were clinically important. Items on the pre-examination checklist included: a past medical history of influenza infection, influenza vaccination status, whether medication had been taken prior to medical visits, awareness of other patients with influenza within close proximity to the study patients (at work, at school, at home, or at another location in close proximity), clinical symptoms (headache, nasal discharge, cough, joint and muscle pain, fatigue, history of fever (acute/sudden or slow)), axillary temperature at home, time of symptom onset, severity of current symptoms (compared to having a common cold), and self-diagnosis presented as a percentage (%). Items collected from patients’ medical records were as follows: age, sex, clinical signs (axillary temperature, pulse rate at clinic), time of RIDT, results of RIDT, and a physician’s final diagnosis.
The self-diagnosis index is a continuous measure, which offers researchers the ability to query the degree to which participants were certain of their diagnosis. This is in juxtaposition to the dichotomous categories of yes and no responses, which do not accommodate the absence of certainty. We used the percentage as a continuous variable for the self-diagnosis index and asked the patient to provide an estimate of the possibility that they were suffering from influenza on the pre-examination checklist.
Diagnosis of influenza
As a reference standard for diagnosing influenza infection, a rapid diagnostic kit (Prime Check Flu, Alfresa Co., Ltd., Japan) was used and laboratory diagnosis was determined by a clinical laboratory technician independent of the physicians attending the patients and independent of the nurses who conducted the pre-examination.
Statistical analysis
ROC curve analysis was performed to estimate the AUC and the optimal cut-off point. The AUC was determined to evaluate the discriminatory power of the self-diagnosis under various conditions28, such as the presence or absence of clinical factors. To calculate Sn, Sp, and the likelihood ratio at each cut-off point, 2 × 2 contingency tables were analysed. To validate the additional contribution of self-diagnosis in predicting the influenza diagnosis, we added self-diagnosis-related information to a multivariate logistic regression analysis including basic clinical signs. We expected the rate of positive influenza testing about 50% with reference to our previous study11, and estimated a sample size of at least 240 based on the minimum 10 events per variable rule for logistic regression analysis29. All statistical analyses were performed with EZR version 1.37 (Saitama Medical Centre, Jichi Medical University, Saitama, Japan), which is a modified version of R Commander that is designed to add statistical functions frequently used in biostatistics30.
Reporting summary
Further information on research design is available in the Nature Research Reporting Summary linked to this article.
Data availability
The data generated and/or analysed during the current study are mostly included in this published article. Additional data are available from the corresponding author upon reasonable request.
References
National Institute of Infectious Diseases. Seasonal influenza in this winter (2017/18 season). https://www.niid.go.jp/niid/images/idsc/disease/influ/fludoco1718.pdf (2018).
Centers for Disease Control and Prevention. Estimated influenza illnesses, medical visits, and hospitalizations averted by vaccination in the United States. https://www.cdc.gov/flu/about/disease/2016-17.htm (2018).
Paules, C. & Subbarao, K. Influenza. Lancet 390, 697–708 (2017).
Molinari, N.-A. M. et al. The annual impact of seasonal influenza in the US: measuring disease burden and costs. Vaccine 25, 5086–5096 (2007).
Putri, W. C., Muscatello, D. J., Stockwell, M. S. & Newall, A. T. Economic burden of seasonal influenza in the United States. Vaccine 36, 3960–3966 (2018).
Chartrand, C., Leeflang, M. M., Minion, J., Brewer, T. & Pai, M. Accuracy of rapid influenza diagnostic tests: a meta-analysis. Ann. Intern. Med. 156, 500–511 (2012).
Mitamura, K. & Sugaya, N. Diagnosis and treatment of influenza-clinical investigation on viral shedding in children with influenza. Uirusu 56, 109–116 (2006).
Monto, A. S., Gravenstein, S., Elliott, M., Colopy, M. & Schweinle, J. Clinical signs and symptoms predicting influenza infection. Arch. Intern. Med. 160, 3243–3247 (2000).
Klepser, M. E. Socioeconomic impact of seasonal (epidemic) influenza and the role of over-the-counter medicines. Drugs 74, 1467–1479 (2014).
Jutel, A. & Banister, E. “I was pretty sure I had the ‘flu”: qualitative description of confirmed-influenza symptoms. Soc. Sci. Med. 99, 49–55 (2013).
Maita, H., Kobayashi, T., Osawa, H. & Kato, H. Self-diagnosis of seasonal influenza in a rural primary care setting in Japan: a cross sectional observational study. PLoS ONE 13, e0197163 (2018).
Jutel, A., Baker, M. G., Stanley, J., Huang, Q. S. & Bandaranayake, D. Self-diagnosis of influenza during a pandemic: a cross-sectional survey. BMJ Open 1, e000234 (2011).
Demicheli, V., Jefferson, T., Ferroni, E., Rivetti, A. & Di Pietrantonj, C. Vaccines for preventing influenza in healthy adults. Cochrane Database Syst. Rev. 2, Cd001269 (2018).
Ginsberg, J. et al. Detecting influenza epidemics using search engine query data. Nature 457, 1012 (2009).
Sueki, H. Relationship between Internet search results and influenza epidemic. J. Health Welf. Stat. 58, 8–13 (2011). (Japanese).
Hesse, B. W. et al. Trust and sources of health information: the impact of the Internet and its implications for health care providers: findings from the first Health Information National Trends Survey. Arch. Intern. Med 165, 2618–2624 (2005).
Zhang, Y., Jones, B., Spalding, M., Young, R. & Ragain, M. Use of the internet for health information among primary care patients in rural West Texas. South Med. 102, 595–601 (2009).
Bashshur, R. L. et al. The empirical foundations of telemedicine interventions for chronic disease management. Telemed. J. E. Health 20, 769–800 (2014).
Ward, M. M., Jaana, M. & Natafgi, N. Systematic review of telemedicine applications in emergency rooms. Int. J. Med. Inf. 84, 601–616 (2015).
Ohannessian, R. Telemedicine: potential applications in epidemic situations. Eur. Res. Telemed./La Rech. Européenne en. Télémédecine 4, 95–98 (2015).
Centers for Disease Control and Prevention. Interim guidance on infection control measures for 2009 H1N1 influenza in healthcare settings, including protection of healthcare personnel. https://www.cdc.gov/h1n1flu/guidelines_infection_control.htm (2010).
Jutel, A. Self-diagnosis: a discursive systematic review of the medical literature. J. Participat. Med. 15, e8 (2010).
Gupta, K., Hooton, T. M., Roberts, P. L. & Stamm, W. E. Patient-initiated treatment of uncomplicated recurrent urinary tract infections in young women. Ann. Intern. Med. 135, 9–16 (2001).
Bent, S., Nallamothu, B. K., Simel, D. L., Fihn, S. D. & Saint, S. Does this woman have an acute uncomplicated urinary tract infection? JAMA 287, 2701–2710 (2002).
Pilger, D., Khakban, A., Heukelbach, J. & Feldmeier, H. Self-diagnosis of active head lice infestation by individuals from an impoverished community: high sensitivity and specificity. Rev. Inst. Med. Trop. Sao Paulo 50, 121–122 (2008).
Abbott, J. Clinical and microscopic diagnosis of vaginal yeast infection: a prospective analysis. Ann. Emerg. Med. 25, 587–591 (1995).
Weakley, J. et al. Agreement between obstructive airways disease diagnoses from self-report questionnaires and medical records. Prev. Med. 57, 38–42 (2013).
Greiner, M., Pfeiffer, D. & Smith, R. Principles and practical application of the receiver-operating characteristic analysis for diagnostic tests. Prev. Vet. Med. 45, 23–41 (2000).
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R. & Feinstein, A. R. A simulation study of the number of events per variable in logistic regression analysis. J. Clin. Epidemiol. 49, 1373–1379 (1996).
Kanda, Y. Investigation of the freely available easy-to-use software ‘EZR’ for medical statistics. Bone Marrow Transpl. 48, 452–458 (2013).
Acknowledgements
The authors would like to thank Itsuki Funakoshi, MD; Shunsuke Soma, MD; and Risa Yamauchi, MD for their clinical support. We also thank the staff of the Rokkasho Centre for Community and Family Medicine and all patients who participated in our study.
Author information
Authors and Affiliations
Contributions
H.M. conceived the study idea. All authors were responsible for the design of the study protocol. H.M., T.K., T.A. and F.M. were responsible for data acquisition and analysis. H.K. and H.O. helped with data management and data analysis. All authors were involved in interpretation of data. H.M. and T.K. wrote the first draft, and all authors critically contributed to discussing and commenting on this paper. H.M. is the guarantor.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Maita, H., Kobayashi, T., Akimoto, T. et al. Factors associated with seasonal influenza self-diagnosis: a prospective observational study in Japan. npj Prim. Care Respir. Med. 30, 9 (2020). https://doi.org/10.1038/s41533-020-0165-3
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41533-020-0165-3