Hemorrhage segmentation in mobile-phone retinal images using multiregion contrast enhancement and iterative NICK thresholding region growing

Hemorrhage segmentation in retinal images is challenging because the sizes and shapes vary for each hemorrhage, the intensity is close to the blood vessels and macula, and the intensity is often nonuniform, especially for large hemorrhages. Hemorrhage segmentation in mobile-phone retinal images is even more challenging because mobile-phone retinal images usually have poorer contrast, more shadows, and uneven illumination compared to those obtained from the table-top ophthalmoscope. In this work, the proposed KMMRC-INRG method enhances the hemorrhage segmentation performance with nonuniform intensity in poor lighting conditions on mobile-phone images. It improves the uneven illumination of mobile-phone retinal images using a proposed method, K-mean multiregion contrast enhancement (KMMRC). It also enhances the boundary segmentation of the hemorrhage blobs using a novel iterative NICK thresholding region growing (INRG) method before applying an SVM classifier based on hue, saturation, and brightness features. This approach can achieve as high as 80.18%, 91.26%, 85.36%, and 80.08% for recall, precision, F1-measure, and IoU, respectively. The F1-measure score improves up to 19.02% compared to a state-of-the-art method DT-HSVE tested on the same full dataset and as much as 58.88% when considering only images with large-size hemorrhages.

A mobile phone with a unique portable retinal lens can conveniently produce many retinal images. Its portability and economic cost attract many health care organizations to use it to prescreen ophthalmic diseases such as glaucoma and diabetic retinopathy (DR) for patients on a large scale 1 . However, mobile-phone retinal images have undesirable characteristics, such as blurry edges, nonuniform illumination, shadowy background, and low contrast.
Hemorrhages are abnormal bleeding from retinal blood vessels. They are one of the prime indicators of DR. The shapes and shades of hemorrhages vary. Common shapes are dome, semilunar, crescentic, plaque, splinter, flame, lozenge, pool, and dot 1,3 . Some may appear as irregular geographic shapes. The colors of the hemorrhages are determined by the levels of the affected retina, leakage amounts, and ages [2][3][4] . Moreover, there can be more than one shade in a single hemorrhage. Hemorrhage detection is generally difficult not only because of wide variations in shapes and shades but also because of the high similarity to blood vessels and shadows in the image. The reviews of work related to hemorrhage detection and segmentation in retinal images are summarized in Table 1. Most work on hemorrhage segmentation completely neglects the fact that a single hemorrhage can have nonuniform intensity, especially large hemorrhages. When a specific intensity range is used for hemorrhage segmentation, incomplete regions are usually segmented, resulting in a low recall value.
The hemorrhage characteristics and the poor image quality produced by a mobile phone make hemorrhage segmentation even more challenging. Light exposure and illumination commonly found in mobile phone retinal images make contrast enhancement ineffective. The shadows directly affect hemorrhage segmentation performance because they are often incorrectly segmented as a hemorrhage, resulting in low precision. Images with www.nature.com/scientificreports/ both shadows, light exposure, and illumination considered poor quality are generally more difficult to detect hemorrhages than good-quality images. This work improves hemorrhage segmentation performance, especially for large hemorrhages with nonuniform intensity, in a poor-quality retinal dataset from a mobile phone.

Objectives and contributions
Improving the hemorrhage segmentation performance for a mobile phone retinal dataset is the main objective of this work. A novel method KMMRC-INRG is proposed. The method considers the problems of uneven background illumination characteristic of a mobile phone retinal dataset and the nonuniform intensity of the hemorrhages. It comprises two new subalgorithms: K-mean multiregion contrast enhancement (KMMRC) and iterative NICK thresholding region growing (INRG). The KMMRC algorithm overcomes the intense illumination and poor contrast in mobile phone images. The INRC algorithm overcomes multishading hemorrhages.
The findings of this work add a new theoretical contribution to the existing knowledge in terms of techniques. The improvement of hemorrhage segmentation performance is a practical contribution. It can directly help improve the DR prescreening and severity grading performances. The proposed algorithms can be used in other applications that involve multishading objects such as an optic disk or skin cancers. Figure 1 shows the framework of KMMRC-INRG. It comprises three main steps: preprocessing, candidate generation, and classification. KMMRC and INRG are novel algorithms proposed in this work. The details of the algorithms are provided in the later sections. The following describes the details of each step. www.nature.com/scientificreports/ A. Preprocessing. As the green channel of the image can help detect better dark components such as hemorrhages 17 , we convert the image's green channel to grayscale and use it as an input. Preprocessing comprises two main tasks. First are the contrast enhancements, which are performed locally and globally. The second is noise removal. Local contrast enhancement is first applied to improve local contrast because the background has areas with different lighting conditions, such as illumination and shadows. We proposed a K-means multiregion contrast enhancement (KMMRC) algorithm to improve the uneven lighting background. The K-means algorithm 18 is used to divide the background into K regions based on intensity. A linear edge enhancement algorithm then enhances the contrast of each region. The pseudoalgorithm is provided in Algorithm 1.

Methodology
The functions used in the algorithm are defined as follows. Our empirical observations show that K = 5 and β = 1.5 give the best performance. We then apply contrastlimited adaptive histogram equalization (CLAHE) 19 to the image to smooth the false edges resulting from local contrast enhancement. To remove the noise and smoothen the image, we apply average filtering 20 to the resultant image.
B. Candidate generation. We used the contrast-enhanced images from the previous step as input. In this step, the hemorrhage candidates from the contrast-enhanced images are created. To obtain the hemorrhage candidates, we performed blob segmentation and vessel removal. The blob segmentation is performed by using the proposed iterative NICK thresholding region growing (INRG) algorithm. www.nature.com/scientificreports/ The INRG algorithm uses the following functions. NT(I) takes an image I as an input and returns a set of (x, y) coordinates of points that pass the NICK threshold in Eq. (1).
where avg x, y is a local average at x, y , V x, y is the intensity at x, y , N is the number of points in the area, and κ is a parameter in the range [−0. Remark ∪ is a union operator. Our empirical observations show that n = 3 and δ = 0.1 give the best performance.
The algorithm first calculates the initial regions of the candidate blobs by searching for pixels that are salient compared to their local backgrounds using NICK thresholding (NT) 21,22 . To better extract a region with nonuniform intensity, the algorithm expands the region by replacing the intensity in the regions with its average. Then, the algorithm reapplies NICK thresholding. It repeats until the region's growth rate is less than a convergence constant. The program repeats at most n-1 times to ensure a complete exit. Figure 2 shows the areas of hemorrhage candidates using the INRG algorithm at different iterations until it converges.
Long and thin blobs are usually vessels. The algorithm detects these blobs by considering the axis length ratio of the fitted ellipse's major and minor axes. From an empirical experiment, the ratio of 6.4 gives the optimal solution; it is assumed to be a blood vessel and is removed.

C. Feature extraction and classification.
The hue-saturation-value (HSV) color space is used in the feature extraction process, as hemorrhages are usually rich in color, have low saturation, and have low brightness. Additionally, HSV is more resistant to external lighting than RGB. The H, S, and V values are used as features extracted from each candidate.
Blob's feature data and correct class answers (hemorrhagic and nonhemorrhagic) are trained and tested using fivefold cross-validation. Figure 3 depicts all processes from the beginning until hemorrhages are obtained.

Dataset and evaluation
We used a retrospective mobile-phone retinal dataset 10 comprising 100 images with a 50-50 ratio of hemorrhagic and nonhemorrhagic images taken by an iPhone 6s with a Volk iNview retinal lens. The data were collected from Thammasat Chalermprakiat Hospital in Thailand in 2019. The images were of type jpg and were all of dimension 598 × 597. Generally, the images had a narrower field of view than retinal images produced from standard ophthalmoscopes. Statistically, there were 32 images with blurry edges, 60 images with light explosive areas, 81 images with shadows, 50 images with uneven illumination, and 8 images with large hemorrhages with nonhomogeneous shades. There were 31 hemorrhagic images in the collection. We evaluate the performance www.nature.com/scientificreports/  www.nature.com/scientificreports/ of KMMRC-INRG in hemorrhage segmentation using standard recall, precision, F1-measure and intersection over union (IoU). The formulas of these evaluations are as follows.
where TP, TN, FP, and FN are the number of blobs that are true positive, true negative, false positive, and false negative, respectively. For image classification, we used a full set of 100 images from the mobile retinal dataset. We used the KMMRC-INRG algorithm to detect hemorrhages. When a hemorrhage was detected in an image, we considered the image positive; otherwise, it was considered negative. We evaluated the performance of the algorithm for classifying images using true positives, true negatives, false positives, and false negatives. The confusion matrix was analyzed and interpreted in terms of sensitivity, specificity, positive predictive value (PPV), and accuracy 10 . The formulas for these evaluations are as follows.
where TP I , TN I , FP I , and FN I are the number of images that are true positive, true negative, false positive, and false negative, respectively.

Results
The results of hemorrhage segmentation of our proposed method were compared against DT-HSVE 10 Table 2.
KMMRC-INRG outperforms all comparative methods in terms of recall. The recall, precision, F1-measure, and IoU of KMMRC-INRG are higher than those of DT-HSVE 10 by 17.92%, 20.24%, 19.02% and 16.96%, respectively. The performances of KMMRC-INRG and its three variants (XKMMRC-NT, XKMMRC-INRG, and KMMRC-NT) are not significantly different. It is worth noting that DT-HSVE outperforms adaptive thresholding, region growing, and the watershed method on the same dataset and ground truth. As KMMRC-INRG outperforms DT-HSVE, it also outperforms all comparative methods of DT-HSVE by implication.
As KMMRC-INRG is designed to improve the detection of large-size hemorrhages, which usually have a nonhomogenous intensity, the performance of KMMRC-INRG depends on the number of images with these characteristics. The higher the value is, the better the improvement. Thus, we consider images with large hemorrhages from the same dataset to see the performance of the proposed method on the targeted images. Table 3 shows the segmentation performance on eight images with large hemorrhages. A hemorrhage is large when the ratio of the hemorrhagic area to the circular area of the retina is greater than 3.5%.
The results of performance comparisons on images with large hemorrhages show that KMMRC-INRG significantly outperforms DT-HSVE. The absolute improvement is as high as 58.88%. KMMRC-INRG improves the recall, F1-measure, and IoU scores of the base model XKMMRC-NT by 9.99%, 7.52%, and 5.02%, respectively. This implies that the two proposed algorithms KMMRC  www.nature.com/scientificreports/   Table 4 show that the proposed work classifies the hemorrhagic images from nonhemorrhagic images very well. The accuracy obtained is as high as 89.00%.
We look at cases that obtained low F1-measure scores to analyze the causes. The reasons are as follows. First, they appear in a light explosive area. The second is because a hemorrhage is within a dark shadow environment. Figure 5 shows two examples of such cases. When a hemorrhage appears in the light explosive area, the algorithm does not repeat because the brightness difference is low. Consequently, the area is undersegmented. The opposite scenario occurs when a hemorrhage is in a dark environment. In this case, the average intensity of each round is low and causes the INRG algorithm to repeat too many times, resulting in oversegmentation. Improving contrast more efficiently is our future work.

Conclusion
The KMMRC-INRG method is proposed in this work. It uses novel K-mean multiregion clustering (KMMRC) to improve the uneven background of mobile-phone retinal datasets. It utilizes a newly proposed method INRG that helps improve the segmentation of a multishade hemorrhage. The SVM is used to classify the candidate hemorrhage blobs based on hue, saturation, and brightness features (HSVs). The KMMRC-INRG method can classify hemorrhagic images from nonhemorrhagic images with up to 89.00% accuracy. It can generally segment hemorrhages with an average F1-measure score of 85.36% on a mobile-phone retinal dataset, which is 19.02% higher than DT-HSVE, the state-of-the-art method. The improvement is even more significant in images with large hemorrhages, which is as much as 58.88%.   www.nature.com/scientificreports/

Data availability
The datasets generated and/or analyzed during the current study are available in Google drive at the following link. https:// drive. google. com/ drive/u/ 0/ folde rs/ 1oK2f HPHxt iPDVa Pa3A1 Iuqua pqC_ KCAd.