External validation for sensitivity of the Ottawa subarachnoid hemorrhage rule in a Japanese tertiary teaching hospital

The Ottawa subarachnoid hemorrhage (OSAH) rule is a validated clinical prediction rule for ruling out subarachnoid hemorrhage (SAH). Another SAH rule (Ottawa-like rule) was developed in Japan but was not well validated. We aimed to validate both rules by examining the sensitivity for ruling out SAH in Japanese patients diagnosed with SAH. We conducted a retrospective cohort study by reviewing the medical records of consecutive adult patients hospitalized with SAH at a tertiary-care teaching hospital in Japan who visited our emergency department between July 2009 and June 2019. Sensitivity and its 95% confidence interval (CI) were estimated for each rule for the diagnosis of SAH. In a total of 280 patients with SAH, 56 (20.0%) patients met the inclusion criteria and were analyzed for the OSAH rule, and a sensitivity of the OSAH rule was 56/56 (100%; 95% CI 93.6–100%). While, 126 (45%) patients met the inclusion criteria of the Ottawa-like rule, and the rule showed a sensitivity of 125/126 (99.2%; 95%CI 95.7–100%). The OSAH rule showed 100% sensitivity among our Japanese patients diagnosed with SAH. The implementation of the Ottawa-like rule should be cautious because the false-negative rate is up to 4%.

www.nature.com/scientificreports/ was estimated to be 3.7 per 100,000 person-years. Over the last three decades, the incidence has declined in Western and other Asian countries, but it has increased by about 60% only in Japan 9 . Thus, it is crucial to avoid a misdiagnosis of SAH, which might have different clinical manifestations in this country. On the other hand, the Ottawa-like rule (positive if any one of the following criteria is present: age ≥ 40 years, neck pain or stiffness, altered level of consciousness, or onset during exertion) was recently proposed by Japanese investigators for the assessment of patients presented emergency department complaining of an acute headache, regardless of a new neurological deficit 10 . As shown in Table 1, the OSAH rule is implemented for very specific populations, which may limit the clinical application, while the Ottawa-like rule seems to be easier to employ than the OSAH rule.
In the current study, we aimed to assess the sensitivity of the OSAH rule and the Ottawa-like rule focusing on Japanese patients diagnosed with SAH.

Materials and methods
Design and setting. This was a retrospective cohort study conducted by reviewing the discharge summary of all consecutive patients who visited the emergency department and were hospitalized and eventually diagnosed with SAH at a tertiary care teaching hospital in Okinawa, Japan in a 10-year period. Data extraction was conducted by two health information managers (HIMs) according to the International Classification of Diseases, 10th Revision codes for subarachnoid hemorrhage from electronic health records (EHRs), and the data were double-checked by the HIMs. Then TS reviewed all the extracted, anonymized discharge summaries and EHRs for confirmation of diagnosis and data gathering. The diagnosis of SAH was confirmed by head CT scan, MRI, or confirmation of red blood cells (> 1 × 10 6 /L) or xanthochromia of cerebrospinal fluid. The ethics research board of the Muribushi Okinawa Project for Teaching Hospitals approved this study (No. 2019-2), and waived written informed consent due to the retrospective nature of the study.
Cohort description. We focused on adult patients (16 years old or older) who were diagnosed with SAH from 2009 to 2019, different from a prospective validation study of the OSAH rule by Perry et al. They recruited consecutive adult patients with nontraumatic acute headache with a peak of intensity presenting within one hour from the onset. Thus, they included patients with acute headaches with or without SAH 6 .
To assess the sensitivity of the Ottawa-like rule, we excluded patients who were referred to us with a definite diagnosis of subarachnoid hemorrhage from other institutions; those who were unconscious of the initial assessment at the emergency department defined as the patient responded to pressure or not in terms of eye-opening, and their verbal responses were vague (words or sounds) or none; those who had sustained head trauma within the past one week; those who had had a similar headache three or more times in the last six months with a confirmed diagnosis or when a patient or his/her representatives opted out of participation in the study.
In addition, to assess the sensitivity of the OSAH rule, we excluded patients whose Glasgow coma scale (GCS) < 15; who had a headache for more than 14 days after onset of headache; those who had already undergone head CT and lumbar puncture and presented for reevaluation; those who had papillary edema on fundoscopy; those who presented with new focal neurological deficits; or those who had known cerebral aneurysm, subarachnoid hemorrhage, brain tumor, ventricular shunt or hydrocephalus, the same as the exclusion criteria of the validation study by Perry et al. 7 .
When one or more of item(s) of each clinical prediction rule (CPR) described in the Table 3 is applicable for the participants, they are considered as at risk of SAH. Outcome measures and statistical analyses. The outcome of interest in the current study was the sensitivity of the OSAH rule and the Ottawa-like rule in patients with SAH and its 95% confidence interval (CI). Data were analyzed using STATA version 15 (Stata Corp LLC, College Station, TX, USA). Patients with missing data were omitted from the analysis. We reported count (proportion) for categorical variables and mean ± standard deviation (SD) for continuous variables, where appropriate. Consent to participate. Patients or the public were not involved in the design, conduct, reporting or dissemination of this study.
Ethics approval. We conducted this study ethically in accordance with the World Medical Association Declaration of Helsinki. This study was an observational study without any intervention to the participants. Thus, we offered patients the opportunity to opt out by posting information about the potential eligible case for this study and the ways to contact the primary investigator.

Results
We found 280 patients aged 16 years or older diagnosed with SAH during the study period. For the validation study of the OSAH rule, 222 patients were excluded. Common reasons for the exclusion were GCS < 15 (n = 118), referred cases after diagnostic confirmation of SAH at other clinics/hospitals (72), new-onset neurological deficits (29), and past history of a cerebral aneurysm (2). Three patients with missing data in the GCS or OSAH rule were also excluded. The remaining 56 patients were included in the study (Fig. 1).
While, we included 126 patients for the validation study of the Ottawa-like rule (Fig. 2). One hundred forty-six patients were excluded, and the reasons for exclusion were the referral case (72), patients with head trauma (3), and unconscious patients at initial presentation to our emergency department (71). Missing data were observed in GCS (7) and Ottawa-like rule (1). Table 2 shows the demographic and clinical characteristics of the patients. Among the patients who validated the OSAH rule, the mean age was 57.5 (SD, 14.4), and there were 39 women (69.6%). Twenty-four patients had hypertension, and 20 were current or past smokers. The mean BMI was 24.2 (SD, 4.5). Median systolic blood pressure was 162 mm Hg (IQR 142-188) mm Hg. Among the patients who were included for the validation    www.nature.com/scientificreports/ As shown in Table 3, the most frequently fulfilled criterion was thunderclap headache (n = 44/46, 95.7%), followed by age ≥ 40 (n = 50/56, 89.3%). The sensitivity of the OSAH rule for the diagnosis of SAH was 56/56 (100%; 97.5% CI 95.4%-100%, one-sided). The Ottawa-like rule in our patients showed a sensitivity of 125/126 (99.2%; 95% CI 95.7%-100%). Misclassification occurred in one patient, a 30-year-old man presented with new right hemi-paresthesia. He underwent MR imaging, which showed a slight SAH; however, an emergency physician missed the finding, and the patient was discharged. Two days later, a radiologist suggested the presence of subarachnoid hemorrhage on MRI. Thus, the patient was called and admitted to our hospital immediately. Additional imaging studies did not reveal the specific etiology of SAH, such as an aneurysm or arteriovenous malformation, and he only needed close observation. Eventually, he was discharged without any disability. Because of the new neurological deficit, he was excluded from the analysis of the OSAH rule.

Discussion
Our results based on Japanese patients diagnosed with SAH show that the OSAH rule has high sensitivity, which is similar to that of the original derivation study and subsequent validation studies in North America and China [6][7][8]11 . Although the current study followed the study in China 8 and was the second validation study in Asia, SAH is recognized with the highest prevalence in Japan 9 ; thus, the validation is most important in this country, and physicians could use the rule to carefully exclude SAH in walk-in patients with headache safely.
The OSAH rule may be useful to rule out SAH for alert and oriented patients with acute peaking pain, although it may be implemented for a very specific population. Including patients with consciousness disturbance to be eligible for rule application, the development of another clinical decision rule for ruling out SAH was attempted in Japan by using the Ottawa-like rule with a sensitivity of 100% (95% CI 98.6-100%) 10 . Our patients showed one misclassification case and low sensitivity based on the Ottawa-like rule. The misclassified case that did not fulfil the Ottawa-like rule had a new hemi-paresthesia, and the patient was not eligible for the analysis of the OSAH rule because of the neurological deficit. The Ottawa-like rule does not exclude patients with focal neurological signs for evaluation; this may not be feasible in clinical practice. In fact, the rule missed a SAH case in our cohort; nevertheless, easier to apply than the OSAH rule.
There were several limitations in the present study that must be considered when interpreting the results. First, this study has a risk of selection bias because we included only patients with SAH in a tertiary teaching hospital and might have failed to include all patients with acute peaking headache. Thus, different from the validation study by Perry et al. 7 , we did not include all patients with acute headache without SAH. Our results focused on the sensitivity of the CPRs among patients with SAH and are at risk of overestimation and the absence of the data about patients with acute headache but not diagnosed as SAH might have weakened the validity of our results because we could not address negative predictive value or negative likelihood ratio of the OSAH rule. Second, we could apply the CPRs for a limited number of patients because of the exclusion criteria, which might have limited the clinical utility and generalizability of the CPRs. Third, this study has a possible risk for underestimation of the sensitivity of the CPRs because of the lack of data involving the CPRs. Data were retrospectively collected from EHRs in the emergency department, and physicians might have potentially failed to document some features of patients, including elements of the OSAH rule because physicians might be occasionally overwhelmed due to some reasons (ex. a severe condition of a patient with SAH, overcrowding or multitasking in an emergency department) and deprived of enough time for the documentation. Finally, this study was based on data from a single-center study with a limited number of eligible cases, so our results might have less generalizability. To overcome these limitations, a multicenter, prospective, large cohort study among patients with acute peaking headache should be planned in Japan. www.nature.com/scientificreports/

Conclusion
In summary, the OSAH rule showed 100% sensitivity among our Japanese patients diagnosed with SAH, while the Ottawa-like rule misclassified one patient with SAH. The implementation of the Ottawa-like rule should be cautious because the false-negative rate is up to 4%, and one misclassification, suggesting low generalizability when the rule is implemented for the larger Japanese population with acute headache.

Data availability
The datasets generated and/or analyzed during the current study are available from the corresponding author on reasonable request.