Stability of MRI radiomic features according to various imaging parameters in fast scanned T2-FLAIR for acute ischemic stroke patients

From May 2015 to June 2016, data on 296 patients undergoing 1.5-Tesla MRI for symptoms of acute ischemic stroke were retrospectively collected. Conventional, echo-planar imaging (EPI) and echo train length (ETL)-T2-FLAIR were simultaneously obtained in 118 patients (first group), and conventional, ETL-, and repetition time (TR)-T2-FLAIR were simultaneously obtained in 178 patients (second group). A total of 595 radiomics features were extracted from one region-of-interest (ROI) reflecting the acute and chronic ischemic hyperintensity, and concordance correlation coefficients (CCC) of the radiomics features were calculated between the fast scanned and conventional T2-FLAIR for paired patients (1st group and 2nd group). Stabilities of the radiomics features were compared with the proportions of features with a CCC higher than 0.85, which were considered to be stable in the fast scanned T2-FLAIR. EPI-T2-FLAIR showed higher proportions of stable features than ETL-T2-FLAIR, and TR-T2-FLAIR also showed higher proportions of stable features than ETL-T2-FLAIR, both in acute and chronic ischemic hyperintensities of whole- and intersection masks (p < .002). Radiomics features in fast scanned T2-FLAIR showed variable stabilities according to the sequences compared with conventional T2-FLAIR. Therefore, radiomics features may be used cautiously in applications for feature analysis as their stability and robustness can be variable.

The paradigm is shifting from qualitative visual assessment of medical imaging to quantitative data analysis with the development of high-throughput mining of low-to high dimensional data. Radiomic features are considered to be an important alternative for interpretation and analysis of medical images and to predict lesion characteristics with numerous features, from first-order to high-order features 1-4 . However, radiomic features can have limitations in their reproducibility or stability. The stability of radiomic features is still challenging with a lack of standardization during image acquisition, reconstruction, segmentation and analyses even though standardized image processing and feature computation have allowed radiomic features to be stable 4 . Among the various types of medical imaging, magnetic resonance imaging (MRI) has a variety of imaging acquisition methods and combinations of complicated parameters even in the same imaging sequences, which makes it difficult to apply radiomic features to MRI.
Fast scanned techniques are essential in the acquisition of MRI because of the major limitation of MRI, the need for a long scan time, particularly in emergency situations such as after a suspected cerebral acute ischemic stroke 5 . Fast scanned images have been realized by using echo-planar imaging (EPI), parallel imaging, echo train length (ETL) and recently introduced advanced techniques such as compressed sensing and simultaneous multi-slice acquisition, and so on [6][7][8][9][10][11][12][13] . The various techniques have resulted in a very complicated combination of imaging parameters, which can hamper the acquisition of stable radiomic features. www.nature.com/scientificreports/ T2-Fluid attenuated inversion recovery (FLAIR) is very commonly used and essential sequence for the evaluation of cerebral acute ischemic stroke patients [9][10][11][14][15][16][17][18][19][20] . Therefore, T2-FLAIR is an important candidate for the application of radiomic features. However, there have been attempts to reduce the scan time of T2-FLAIR for a long time, which resulted in various parameters of T2-FLAIR. The EPI, parallel imaging, and ETL have been widely used 6,[9][10][11][12] . Nevertheless, the stability of the radiomic features have been poorly investigated.
We hypothesized that radiomic features from fast scanned T2-FLAIR show variability relative to conventional T2-FLAIR. Therefore, the aim of our study was to investigate the stability of radiomic features from various fast scanned T2-FLAIR images in patients with acute ischemic stroke, and to compare the agreement of the radiomic features with conventional T2-FLAIR as a reference standard.

Discussion
This study showed a consistent tendency of higher proportions of reliable features in EPI-T2-FLAIR and TR-T2-FLAIR than ETL-T2-FLAIR in both acute and chronic ischemic hyperintensities and for both whole-and intersection-ROI mask. Therefore, various image acquisitions of T2-FLAIR resulted in unstable radiomic features, which may lead to different radiomic features' outcomes, such as prediction modeling.
MRI is a useful and sometimes essential imaging modality to identify the infarct core on DWI, and additional useful information can be obtained from various images such as T2-FLAIR or gradient echo (GRE) images, and also allow acquisition of vessel information without the need for contrast media during the evaluation of cerebral acute ischemic stroke patients; however, MRI has a lesser availability and a longer scan time compared to CT 5,21-25 . Therefore, there have been many attempts to reduce the scan time of MRI in cerebral acute ischemic stroke situations, which has resulted in various MRI sequences and parameters being applied in clinical practice 6,[9][10][11][12] .
Previous studies on fast-scanned T2-FLAIR in acute ischemic stroke showed a consistent superior reliability when compared with that of conventional images 6,[9][10][11] . However, those studies only showed repeatability or reliability in qualitative scoring systems or simple quantitative comparisons, such as signal intensity. In contrast, our study based on radiomic features showed a lower reliability than that of conventional images even though the data was originated from the same registry in the previous study 6 .
The diversity in the image acquisition makes it difficult to apply radiomic features to MRI for cerebral acute ischemic stroke. Imaging acquisition, segmentation, and feature extraction can affect the stability of radiomic features 2 . Ford et al. demonstrated that changes of imaging parameters could lead to variable radiomic features in a phantom study 26 . Minjae et al. only showed that the change in acceleration factors on the same images can affect the stability of radiomic features, and two different under-sampling methods on the same images can show different radiomic features even under the same acceleration factors 27 . Therefore, different imaging parameters even on the same FLAIR sequences may reduce the stability of radiomic features, as in this study. In addition, these obstacles may affect a few published studies on the predictive models developed using MRI radiomic features in cerebral acute ischemic stroke [28][29][30] . The results from this study showed some stable radiomic features across variable acquisition of T2-FLAIRs and acute and chronic ischemic hyperintensities, but a substantial proportion showed variability. To our knowledge, there is no previous report on the stability of MRI radiomic features according to various imaging parameters in cerebral acute ischemic stroke. The segmentation can affect the stability of radiomic features. Manual segmentation, compared with semi-or automated segmentations, may lead to lower reproducibility in radiomic features 2 . However, many previous studies on radiomic features or high dimensional quantitative analyses using artificial intelligence relied on manual segmentations. In addition, Haarburger et al. showed poor reproducibility of some radiomic features even under automated segmentation methods 31 . The segmentation reproducibility can be influenced by the anatomic location and lesion types 27,32 . In our study, there were some different results between whole and intersection ROI masks, which may be also owing to different sizes of ROI masks and thus different numbers of pixels. Feature extraction can also affect the stability of radiomic features. Li et al. demonstrated the poor stability of radiomic features (no features > 0.85 in concordance correlation coefficient [CCC]) across different extraction combinations 33 .
Studies evaluating stable radiomic features in cerebral acute or chronic ischemic lesions based on multiparameteric variances appear to be lacking, and the reproducibility of radiomic features in brain tumors, including glioblastoma, has been reported and several features belonging to GLRLM were identified as reproducible features 33,34 . GLRLM was also the most reproducible feature in cine balanced steady-state free procession and first-order, and GLCM had the most reproducible features on both T1 and T2 maps in the myocardial radiomic features 35 . However, a phantom study for test-retest reproducibility reported that GLRLM was neither the most robust nor least robust feature class, while GLCM was one of the least robust feature classes across MRI sequences: FLAIR, T1-weighted, and T2-weighted imaging 36 . Our study also showed that GLCM and GLRLM are common stable features in the numbers but some variability was seen in the proportions of stable radiomic features.
This study has several limitations. First, this study was designed as a retrospective study with a small population in a single center. A study population cannot be free from selection bias, which may have affected the deviations in sex and age, and a specific MR machine was adopted. Therefore, further multi-center studies with www.nature.com/scientificreports/ a larger sample size are necessary. Second, we did not compare all of the T2-FLAIRs simultaneously because it is hard to obtain all of the T2-FLAIRs at the same time in a cerebral acute ischemic stroke situation. Third, this study presented only the stabilities of radiomic features in T2-FLAIRs of acute ischemic stroke and an evaluation of the stability of identification or prediction models influencing the treatment options for stroke outcomes using radiomic features from variable T2-FLAIRs is necessary.
In conclusion, the fast-scanned T2-FLAIR showed small proportions of stable radiomic features and variable stability of radiomic features among those in terms of the agreements with conventional T2-FLAIR. Therefore, even if the images in the same sequence have different parameters, the radiomic features obtained from the images may be used with caution.

Methods
Study population. From May 2015 to June 2016, data on 296 patients undergoing MRI at a single tertiary hospital for symptoms of acute ischemic stroke were retrospectively collected. Among them, 118 patients underwent echo-planar imaging (EPI)-T2-FLAIR and echo train length (ETL)-T2-FLAIR and 178 patients underwent ETL-T2-FLAIR and repetition time (TR)-T2-FLAIR simultaneously. In total, 79 patients showed acute ischemic hyperintensity and 89 patients showed chronic ischemic hyperintensity on simultaneous acquisition of EPI-and ETL-T2-FLAIR, who were classified to the first group, and 112 patients showed acute ischemic hyperintensity and 127 patients showed chronic ischemic hyperintensity on simultaneous acquisition of ETLand TR-T2-FLAIR, who were classified to the second group for comparable paired data analysis. The detailed demographics of the patients are presented in Table 2. The data on patients were collected from the fast stroke MRI registry in our institute 6 . The institutional review board of Asan Medical Center approved the present study, and the requirement for informed consent was waived. The data was analyzed in compliance with the International Council for Harmonization of Technical Requirements for Registration of Pharmaceutical for Human Use: Guideline for Good Clinical Practice (ICH GCP).
Image acquisition. All T2-FLAIR was scanned on a 1.5-T scanner (Magnetom Avanto; Siemens Healthineers). The scan times were 128 s for conventional T2-FLAIR, 45 s for EPI-T2-FLAIR, 74 s for ETL-T2-FLAIR, and 79 s for TR-T2-FLAIR. The detailed scan parameters for the conventional and fast T2-FLAIR were as previously reported 6 and are listed in Table 3 and representative images are presented in Fig. 4.

Image analysis.
The segmentations of acute-and chronic ischemic hyperintensities were conducted as described in a previous report 6 . We defined acute ischemic hyperintensity as a T2-FLAIR high signal intensity within acute infarcts on diffusion weighted images (DWI) 15,17 , and chronic ischemic hyperintensity as a hyperintensity outside of acute infarcts on DWI. The segmentation of region-of-interest (ROI) mask was done Numbers of stable radiomic features from whole ROI masks (c) and intersection ROI masks (d) according to each fast scanned T2-FLAIR. AIH acute ischemic hyperintensity, CIH chronic ischemic hyperintensity, ROI region-of-interest, EPI echo-planar imaging, ETL echo train length, TR repetition time, GLCM gray-level co-occurrence matrix, GLRLM gray-level run-length matrix, LBP local binary pattern, GLSZM gray-level size zone matric, NGTDM neighboring gray tone difference matrix.  . (a,b) Proportions of stable radiomic features from whole ROI masks (a) and intersection ROI masks (b). (c,d) Numbers of stable radiomic features from whole ROI masks (c) and intersection ROI masks (d). AIH acute ischemic hyperintensity, CIH chronic ischemic hyperintensity, ROI region-of-interest, EPI echo-planar imaging, ETL echo train length, TR repetition time, GLCM gray-level co-occurrence matrix, GLRLM gray-level run-length matrix, LBP local binary pattern, GLSZM gray-level size zone matric, NGTDM neighboring gray tone difference matrix. www.nature.com/scientificreports/ by one research assistant (K.M.C. with 5 years of experience in stroke imaging) using an in-house program for semiautomatic segmentation (based on ImageJ software; National Institutes of Health, Bethesda, MD). The intersection ROI mask between the respective ROI mask of the conventional and fast T2-FLAIR was obtained after the coregistration and then each intersection ROI mask was transferred into the respective conventional and fast T2-FLAIR images. The intersection ROI mask was used to compare the radiomic features from different T2-FLAIRs without ROI mask differences. A total of 14 kinds of ROI masks were obtained as follows: 3 ROIs from conventional-, EPI-, and ETL-T2-FLAIR in the first group; 2 intersection ROIs from conventional and EPI-T2-FLAIR, and 2 intersection ROIs from conventional and ETL-T2-FLAIR in the first group; 3 ROIs from conventional-, ETL-, and TR-T2-FLAIR in the second group; 2 intersection ROIs from conventional and ETL-T2-FLAIR, and 2 intersection ROIs from conventional and TR-T2-FLAIR in the second group (Fig. 5). From the ROIs, 595 radiomics features were extracted and concordance correlation coefficients (CCC) for radiomic features were calculated between fast scanned and conventional T2-FLAIR in each group. Statistical analysis. The stability of the radiomic features was evaluated using CCC between the features extracted from the conventional-and fast scanned T2-FLAIR based on Lin's definition 40

Data availability
The datasets collected during and/or analyzed during the current study may be available from the corresponding author on reasonable request and in compliance with ethical standards under an approval of the local institutional review board. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/.