Effects of contrast-enhancement, reconstruction slice thickness and convolution kernel on the diagnostic performance of radiomics signature in solitary pulmonary nodule

The Effects of contrast-enhancement, reconstruction slice thickness and convolution kernel on the diagnostic performance of radiomics signature in solitary pulmonary nodule (SPN) remains unclear. 240 patients with SPNs (malignant, n = 180; benign, n = 60) underwent non-contrast CT (NECT) and contrast-enhanced CT (CECT) which were reconstructed with different slice thickness and convolution kernel. 150 radiomics features were extracted separately from each set of CT and diagnostic performance of each feature were assessed. After feature selection and radiomics signature construction, diagnostic performance of radiomics signature for discriminating benign and malignant SPN was also assessed with respect to the discrimination and classification and compared with net reclassification improvement (NRI). Our results showed NECT-based radiomics signature demonstrated better discrimination and classification capability than CECT in both primary (AUC: 0.862 vs. 0.829, p = 0.032; NRI = 0.578) and validation cohort (AUC: 0.750 vs. 0.735, p = 0.014; NRI = 0.023). Thin-slice (1.25 mm) CT-based radiomics signature had better diagnostic performance than thick-slice CT (5 mm) in both primary (AUC: 0.862 vs. 0.785, p = 0.015; NRI = 0.867) and validation cohort (AUC: 0.750 vs. 0.725, p = 0.025; NRI = 0.467). Standard convolution kernel-based radiomics signature had better diagnostic performance than lung convolution kernel-based CT in both primary (AUC: 0.785 vs. 0.770, p = 0.015; NRI = 0.156) and validation cohort (AUC: 0.725 vs.0.686, p = 0.039; NRI = 0.467). Therefore, this study indicates that the contrast-enhancement, reconstruction slice thickness and convolution kernel can affect the diagnostic performance of radiomics signature in SPN, of which non-contrast, thin-slice and standard convolution kernel-based CT is more informative.

which is the practice of processing high-throughput extraction of quantitative features to convert images into mineable data for decision support 13 , has been proposed to noninvasively decode tumor phenotype [14][15][16] . Among the diagnostic objective features of SPN, CT-based texture analysis could effectively differentiate benign from malignant lesions 17,18 . However, a recent study by Dennis et al. indicated that the inter-scanner differences existing among different CT scanner could affect the variability in the values of radiomics features 19 . Being the most common varied factors in clinical settings on the imaging modality, whether the imaging acquisition parameters of contrast-enhancement, reconstruction slice thickness and convolution kernel could affect the diagnostic performance of radiomics features on the differential diagnosis of SPN is an interesting field that has been explorated 13,20 . Although individual CT texture feature is useful in the characterization of SPN 17,18,21 , integrating multiple features into a predictive panel as a radiomics signature may be a robust approach for quantifying tumor phenotype 22,23 . Thus, regarding the influence of scanning parameters on the individual feature performance in the previous studies 13,19,24 , radiomics signature could consequently make impact on the performance of differential diagnosis of SPN.
Therefore, the purpose of this study was to investigate the effects of contrast-enhancement, reconstruction slice thickness and convolution kernel on the differential diagnosis performance of radiomics signature in SPN, and to determine the optimal imaging parameters (contrast-enhancement, reconstruction slice thickness and convolution kernel) for extracting radiomics features.

Patients. The retrospective study was approved by the Research Ethics Committee of Guangdong General
Hospital, Guangdong Academy of Medical Sciences (protocol No. GDREC2015192H). Due to the retrospective nature of the study, our institutional review board approved the review of patient data before its commencement and waived the requirement for informed consent. The institutional database was evaluated to collect a primary cohort of this study from January 2010 to December 2012. Patients with biopsy-or surgery-proven primary lung malignancy or benign lesions were enrolled. From January 2013 to July 2015, patients who met the same criteria were included to form an independent validation cohort. Baseline clinical data including age and gender were recorded, and the dates of baseline CT imaging were also recorded.
CT Image Acquisition. All patients underwent non-contrast and contrast-enhanced CT with a multi-detector row CT (GE Light-speed Ultra 8; GE Healthcare, Hino, Japan). Contrast-enhanced CT image was performed after 25 s delay following intravenous administration of 85 ml of iodinated contrast material (Ultravist 370, Bayer Schering Pharma, Berlin, Germany) at a rate of 3.0 ml/s with a pump injector (Ulrich CT Plus 150, Ulrich Medical, Ulm, Germany) after routine non-contrast CT. The fixed acquisition parameters were as follows: 120 kV; 160 mAs; 0.5-or 0.4-second rotation time; detector collimation: 8 × 2.5 mm or 64 × 0.625 mm; field of view, 350 × 350 mm; matrix, 512 × 512. Each patient of the study had four sets of chest CT images with different imaging parameters of contrast-enhancement parameter (non-contrast or contrast-enhancement), reconstruction slice thickness (5 mm or 1.25 mm) and convolution kernel (standard or lung), which were separately labeled as group 1 (non-contrast + 1.25 mm + standard convolution kernel), group 2 (contrast enhancement + 1.25 mm + standard convolution kernel), group 3 (non-contrast + 5 mm + standard convolution kernel) and group 4 (non-contrast + 5 mm + lung convolution kernel). Generation of CT images utilizing different convolution kernels can optimize lesion detection. Lung convolution kernel is generated when high-pass filter algorithm is used, with high spatial frequencies and noise preserved; while low-pass algorithm enables the generation of standard kernel image, with high spatial frequency contribution and noise decreased. The lung convolution kernel and standard convolution kernel are dependent on the vendor of GE.
Radiomics feature extraction. All 4 sets of the CT images were used for radiomics feature extraction, after retrieved from the picture archiving and communication system (PACS) (Carestream, Canada). In-house feature extraction algorithm was implemented in Matlab 2014a (Mathworks, Natick, USA). In total, 150 radiomics features which covered the category of gray-level histogram and gray-level co-occurrence matrix (GLCM) were extracted from each set of CT image. A region of interest (ROI) was delineated initially around the tumor outline for the largest cross-sectional area on each set of CT images. Figure 1 demonstrates the ROI delineations for 2 patients who have malignant and benign tumor respectively, with their size and diameter being listed Supplementary Table S1, respectively. The ROI was further refined by excluding air area with a threshold that removed from analysis any pixels with attenuation values below − 50 HU and beyond 300 HU. A Laplacian of Gaussian spatial band-pass filter (∇ G 2 ) was used to derive image features at different spatial scales by turning the filter parameter between 0 and 2.5 (0, 1.0, 1.5, 2.0, 2.5). The Laplacian of Gaussian filter (∇ G 2 ) distribution is given by x, y denote the spatial coordinates of the pixel and σ is the value of filter parameter. The feature extraction algorithms are described in Supplementary Method S1, and series of gray-level histogram and gray-level co-occurrence matrix (GLCM) features derived were also listed in Table 1.
Intra-reader reproducibility of radiomics features. Intra-reader reproducibility of radiomics feature extraction was initially analyzed with 40 randomly chosen patients (30 malignant and 10 benign) for ROI delineation. The same radiologist who has 10 years' experience in chest CT interpretation repeated the generation of radiomics features twice in a 1-week period followed the same procedure. The size and diameter of each Scientific RepoRts | 6:34921 | DOI: 10.1038/srep34921 delineated ROI were measured and recorded. Intra-class correlation coefficients (ICCs) were used to evaluate the intra-reader agreement of the size and diameter of the tumor and each of the 150 radiomics features extracted from the delineated ROIs with a value greater than 0.75 indicating good intra-reader agreements 25 .

Statistical analysis.
All statistical analysis in this study was conducted with R software, version 3.2.1 (http:// www.Rproject.org).
Differences in age, gender of patients between benign and malignant in both the primary and validation cohort were compared by using the independent sample t test or the Mann-Whitney U test, the Chi-Squared test or the Fisher exact test, where appropriate. And the same tests were also applicable for the assessment of difference in patients' age, gender between primary and validation cohort.
Diagnostic performance of radiomics features. The association of the radiomics features on discrimination between benign and malignant SPN in both primary cohort and validation cohort across different sets of CT images was assessed using Mann-Whitney U test due to its non-normal distribution. Then, the diagnostic performance of the radiomics features was assessed with respect to the area under the curve (AUC) of the receiver operating characteristic curve (ROC). An AUC of 1 indicates perfect discrimination, and random guess gives an AUC of 0.5.
Feature selection and radiomics signature building. Based on the results of univariate analysis of radiomics features, feature selection and data dimension reduction were done using least absolute shrinkage and selection operator method (LASSO) logistic regression model 26 to select the most useful prognostic features of all the associated radiomics features identified with the primary cohort. The LASSO, which is suitable for the regression of high dimensional data using the "glmnet" package in R software, is a penalized estimation technique in which the estimated regression coefficients are constrained so that the sum of their scaled absolute values falls below some constant k chosen by cross-validation. This kind of constraint forces some regression coefficient estimates to be exactly zero, thus achieving variable selection while shrinking the remaining coefficients toward zero to reflect the overfitting caused by data-based model selection. The radiomics signature was built for each patient in both the primary and the validation cohort through the linear combination of features selected by their respective coefficients, with a radiomics score calculated for each patients. A larger score indicates a higher probability to be malignant.

Diagnostic performance and comparison of radiomics signature derived from different CT sets.
The potential association of radiomics signature on discrimination between benign and malignant SPN was also assess using Mann-Whitney U test. The diagnostic performance of radiomics signature was assessed in terms of discrimination and classification. ROC curves for each group dataset were constructed and the area under the curves (AUC) were calculated with histopathological diagnosis of SPNs as outcome. Sensitivity, specificity, and accuracy were also derived as the methods of classification measurement.
For the comparison of discrimination ability for radiomics signatures on diagnostic performance in SPN, the nonparametric test of Delong test was used for comparing the difference in AUC of ROC between groups 27 . A two-sided P value less than 0.05 was considered to indicate the statistical significant difference. A net reclassification improvement (NRI) calculation which is regarded as an increasingly popular measure for evaluating improvements in risk predictions [28][29][30] was also applied for assessing whether one group of prediction performance is better than another. The formula for calculating the NRI (Net Reclassification Index): In this formula, upward movement (up) was defined as a change into higher category based on the new biomarker and downward movement (down) as a change in the opposite direction. The value of NRI can either be positive or negative. A positive value of NRI derived in this study indicates a net improvement in risk classification for patients with SPN.
Finally, the same comparison for each group of radiomics signatures was assessed in the independent validation cohort.

Results
Clinical characteristics and distribution of patients. In total, we retrospectively identified 240 consecutive patients with SPN (benign, n = 60, such as hamartoma (33), pulmonary crytococcosis (5), inflammatory pseudotumor (5), inflammatory granuloma (10), pulmonary sclerosing hemangioma (7); malignant, n = 180, such as lymphoepithelioma (6), squamous-cell carcinoma (22), adenocarcinoma (145), metastatic tumor (7)) between January 2010 and July 2015 who underwent chest CT as the whole study cohort. 120 cases of the institutional database from January 2010 to December 2012 were identified as the primary cohort and other 120 cases Kurtosis describes the sharpness of the histogram = Skewness describes the degree of asymmetry around the mean value in the gray level histogram = Contrast measures local intensity variation, reflects the uniformity of image grayscale distribution and the degree of Correlation measures the gray level linear dependence between the pixels at the specified positions relative to each other  Table 1. Feature extraction algorithms and lists of features derived. Note: X(i) indicates the intensity of gray level i; N denotes the sum of pixels in the image; β indicates the top percentage of the histogram curve, which could be 50%, 25%, and 10%; M denotes the number of pixels in the histogram on the percentage of (1 − β ); x, y denote the spatial coordinates of the pixel; P(i, j) is the co-occurrence matrix by the δ = 1 and θ (0°, 45°, 90°, 135°); N g denotes the number of discrete intensity levels in the image; μ is the mean of P(i, j); μ x (i) is the mean of P x (i); μ y (j) is the mean of P y (j); σ x (i) is the standard deviation of P x (i); σ y (j) is the standard deviation of P y (j). σ represents the filter value applied, which could be 0, 1.0, 1.5, 2.0 and 2.5. α represents the considered direction, which could be 0°, 45°, 90°, and 135°. β represents the top percentage of the histogram curve, which could be 50%, 25%, and 10%. from January 2013 to July 2015 were identified as the validation cohort, respectively. Distribution of patients' characteristics in both primary and validation cohorts between the benign group and malignant group are presented in Table 2. The patients in benign group were younger than that in malignant group (p < 0.001 for both primary and validation cohort), but there was no significant difference in gender between benign and malignant group in both primary (p = 0.745) and validation cohort (p = 0.832). No difference was found between the primary and the validation cohort in the clinical characteristics (p = 0.253 for age, and p = 0.514 for gender).
Intra-reader reproducibility of radiomics features. Satisfactory intra-reader reproducibility of ROI delineation for the 4 groups was achieved, with an intra-class correlation coefficient (ICC) of 0.795 and 0.802 for the size and diameter, respectively. The ICCs for 150 radiomics features had been listed in the Supplement Table S2, ranged from 0.752 to 1.000.
Diagnostic performance of radiomics features. There were 66, 52, 39 and 62 features which showed significant association between the radiomics features and the status of SPN in group 1, group 2, group 3 and group 4, respectively (p < 0.05) (Fig. 2). The univariate analysis between each of the significant associated radiomics features were presented in Fig. 2 and listed in Supplementary Table 3 Supplementary Table S4. The corresponding radiomics signature score calculation formula was presented in the Supplementary Equations S1-S4. The number of selected features varied greatly among 4 groups. In addition, the categories of selected features also varied across different radiomics signatures.
Diagnostic performance of radiomics signature derived from different groups. There was significant difference in radiomics signature scores between benign and malignant patients for four groups in primary cohort (p < 0.001), which was consistent with the validation cohort (p < 0.001). Malignant patients generally had higher scores in both the primary cohort and validation cohort ( Table 2). The distribution of radiomics signature scores for classification of SPN status in the primary cohort and validation cohort are shown in Fig. 3. Further, the diagnostic performance of radiomics signature varied greatly in both the primary and validation cohort across all 4 groups with a varied AUC of 0.686-0.862, sensitivity of 0.667-0.944, specificity of 0.533-0.867 and accuracy of 0.708-0.858 (Table 3). Table 3, there was significant variability in the diagnostic performance of radiomics signatures in SPN based on features extracted from CT images acquired with different parameters (contrast-enhancement, reconstruction slice thickness and convolution kernel). Although AUCs were different between groups in both primary cohort and validation cohort, the NRIs were also analyzed (

Discussion
This study demonstrated that incorporating individual radiomics features extracted from CT images as a radiomics signature facilitated the differential diagnosis of SPN, and the variability of acquisition parameters (contrast-enhancement, reconstruction slice thickness and convolution kernel) had effects on the diagnostic performance of radiomics signature in SPN. In addition, we also demonstrated that radiomics signature based on CT images acquired with non-contrast, thin-slice and standard convolution kernel had better performance on the differential diagnosis of SPN. As one processes of radiomics studies, optimum protocols for image acquisition and reconstruction algorithm have to be identified and harmonized 13,20 . The variation caused by different parameters implies that it should be possible to make consistency for this acquisition parameters used in radiomics studies 13,19 . To date, most studies of radiomics features have focused on finding robustness features 19,31-33 . Among the above studies, Leijenaar et al. studied the stability of radiomics features 32 , and Hunter et al. identified the high quality machine-robust image features 33 . Although robustness radiomics features were presented in the above previous study, it's worthy of notice that the impact of acquisition parameters on radiomics signatures could vary widely, which has never been investigated. As expected, our study showed that the contrast-enhancement, reconstruction slice thickness and convolution kernel affected the diagnostic performance of radiomics features as revealed by the univariate analysis, and as well as the corresponding radiomics signatures constructed by LASSO regression method in SPN.
Radiomics signature built based on the non-contrast CT images showed better performance in the differential diagnosis of SPN, compared with the contrast-enhanced CT images. The underlying reason for the better performance on non-contrast images may be that the biological heterogeneity within the tumor that can be depicted by radiomics features may be confounded by the intravenous injected contrast material, which may then skewness  kurtosis  contrast_0  contrast_45  contrast_90  contrast_135  correlation_0  correlation_45  correlation_90  correlation_135  energy_0  energy_45  energy_90  energy_135  homogeneity_0  homogeneity_45  homogeneity_90  homogeneity_135  entropy_0  entropy_45  entropy_90  entropy_135  his_mean  his_SD  his_50_mean  his_50_SD  his_25_mean  his_25_SD  his_10_mean  his_10_SD   0    result in poorer discrimination between malignant and benign tumors due to the existing intratumoral contrast material [34][35][36] .
Regarding the reconstruction slice thickness of CT images, the radiomics signature built based on thin slice thickness (1.25 mm) was found to have better performance in the differential diagnosis of SPN as compared with that built on thick slice thickness (5 mm) in our study. Similarly, previous studies had presented that slice thickness could significantly affect the quantification of CT image features, and illustrated that slice thickness of 1.25 mm and 2.5 mm were better than 5 mm for texture features 37 . The underlying reason for the better performance of thin-slice images may be that thicker slice images introduce larger partial pixel artifacts as compared to thinner slice images 37,38 . Furthermore, we found that radiomics signature built based on standard convolution kernel CT had better diagnostic performance than that built based on lung convolution kernel CT. The underlying reason for the better performance on standard convolution kernel images may be that generation of CT images utilizing different convolution kernels can optimize lesion detection. Lung convolution kernel is generated when high-pass filter algorithm is used, with high spatial frequencies and noise preserved; while low-pass algorithm enables the generation of standard kernel image, with high spatial frequency contribution and noise decreased, and work best for tissues with inherently lower contrast, such as lung tissues 39 .
As discussed above, the acquisition parameters (contrast-enhancement, reconstruction slice thickness and convolution kernel) affected the features selection (12, 4, 3 and 3 features selected out of the significant associated radiomics features in group 1, group 2, group 3 and group 4, respectively), with which the corresponding radiomics signature was constructed. Accordingly, the variability of radiomics signature demonstrates different diagnostic performance in SPN. Our results showed that radiomics signature based on the non-contrast, thin-slice and standard convolution kernel-based CT was more informative on differential diagnosis of SPN.
Limitation of this study includes the fact that the variability in radiomics signature on differential diagnosis of SPN could be caused by different types of CT scanners. All sets of images in our study were generated by the same CT scanner. Among the previous study, Dennis et al. found that the inter-scanner differences on the variability in the values of radiomics features should be considered 19 . Therefore, there might be an interesting attempt to explore the effects of different CT inter-scanners on the differential diagnostic performance of radiomics signature on differential diagnosis of SPN in future studies. Another limitation of this study includes the fact that the dataset may be skewed, with which 180 patients with malignant cancer and only 60 patients with benign tumor composed the whole study cohort. However, our study consists of all consecutive solitary pulmonary nodules that were biopsy-or surgery-proven malignant or benign nodules in our institution collected from January 2010 to July 2015. All images were collected from patients scanned with the same scanner, since there is variability in the quality and repeatability of radiomics features between CT scanners observed in a previous study 19 . Although the dataset was skewed with limited benign cases, the incidence ratio between benign and malignant cases was representative of the intended population in clinical practice. As noted in the TRIPOD statement (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis) 40 , selectively choosing or omitting participants may cast doubt on the representativeness of the sample to the population in which the marker or model is to be applied and affect its generalizability. So despite that ideally the inconsistence should be minimized to reduce the impact on the comparison of the radiomics signature performance, our study enrolled all consecutive patients eligible to the generalizability.
In conclusion, this study presents that the contrast-enhancement, reconstruction slice thickness and convolution kernel can affect the diagnostic performance of radiomics signature in SPN, of which non-contrast, thin-slice and standard convolution kernel-based CT is more informative. The impact of different CT image acquisition parameters on the performance of radiomics signatures should be considered in the future radiomics studies in SPN.  Table 3. Diagnostic performance of discrimination and classification of radiomics signature. Note: 95%CI: 95% confidence interval. AUC: area under curve. SEN: sensitivity; SPE: specificity. Group 1 = noncontrast + 1.25 mm + standard convolution kernel; Group 2 = contrast enhancement + 1.25 mm + standard convolution kernel; Group 3 = non-contrast + 5 mm + standard convolution kernel; Group 4 = noncontrast + 5 mm + lung convolution kernel.  Table 4. NRI of inter-group comparison for the primary cohort and validation cohort. Note: NRI = Net Reclassification Improvement; NRI Events = Net Reclassification Improvement for events; Non-NRI Events = Net Reclassification Improvement for non-events. Group 1 = non-contrast + 1.25 mm + standard convolution kernel; Group 2 = contrast enhancement + 1.25 mm + standard convolution kernel; Group 3 = non-contrast + 5 mm + standard convolution kernel; Group 4 = non-contrast + 5 mm + lung convolution kernel.