Background & Summary

With the advent of brightfield and fluorescent digital scanners that produce and store whole slide images (WSIs) in digital form, there is a growing trend to exploit computerized methods for semi or fully-automatic WSI analysis1. In digital pathology and biomedical image analysis, nuclei segmentation plays a fundamental role in image interpretation2. Specific nuclei characteristics such as nuclei density or nucleus-to-cytoplasm ratio can be used for cell and tissue identification or for diagnostic purposes such as cancer grading2,3,4. Nuclei instance segmentation masks enable the extraction of valuable statistics for each nucleus5. While experts can manually segment nuclei, this is a tedious and complex procedure as thousands of instances can appear in a small patch of a WSI4,6. It is also worth mentioning that due to various artifacts such as folded tissues, out-of-focus scanning, considerable variations of nuclei staining intensities within a single image, and the complex nature of some histological samples (e.g., high density of nuclei), accurate and deterministic manual annotation is not always possible, even for human experts. The inter- and intraobserver variability reported in previous studies showing a low level of agreement in the annotation of cell nuclei by medical experts confirms this general problem5,7.

In recent years, many semi- and fully-automatic computer-based methods have been proposed to perform nuclei instance segmentation automatically and more efficiently. A wide range of approaches from classical image processing to advanced machine learning methods have been proposed for this task4,7. Up to this point, supervised deep learning (DL) methods such as Mask R-CNN and its variants8,9, distance-based methods10,11 and multi encoder-decoder approaches6,12,13 have shown the best instance segmentation performances. However, to train these models, fully annotated datasets are required which is difficult to acquire in the medical domain4,5,14.

A number of fully annotated nuclei instance segmentation datasets are available. These datasets were introduced for various types of staining such as Hematoxylin and Eosin (H&E), immunohistochemical and immunofluorescence stainings4,15,16,17. The most common staining type in routine pathology is H&E-staining. Therefore, most introduced datasets were based on this staining method. Although these datasets are valuable contributions to the research field and help researchers to develop better segmentation models, providing more annotated datasets from different organs and centers to cover more data variability is still of high importance. Table 1 shows the most prominent fully manually annotated H&E-stained nuclei segmentation datasets that have been actively used by the research community in the past few years. Besides these datasets, some semi-automatically generated datasets such as PanNuke18, Lizard (used in the CoNIC challenge)19,20, and Hou et al. dataset21 have also been introduced in the past. To generate these datasets, various approaches, such as using trained backbone models or point annotation, were exploited20,22,23. However, training models based on semi-automatically generated datasets may introduce a hidden bias towards the reference model instead of learning the true human expert style for nuclei instance segmentation. It is also worth mentioning that other publicly available datasets, such as the OCELOT challenge dataset24 or the DigestPath dataset25, have been introduced for different nuclei analysis tasks, including nuclei detection and classification. However, the primary focus of this study is on the task of nuclei instance segmentation.

Table 1 Publicly available H&E-stained nuclei segmentation datasets.
Full size table

In this paper, we present NuInsSeg26, one of the most comprehensive, fully manually annotated, publicly available datasets for nuclei segmentation in H&E-stained histological images. The primary statistic of this dataset is presented in the last row of Table 1. Our dataset can be used alone to develop, test, and evaluate machine learning-based algorithms for nuclei instance segmentation or can be used as an independent test set to estimate the generalization capability of the already developed nuclei instance segmentation methods27.

Methods

Sample preparation

The NuInsSeg dataset26 contains fully annotated brightfield images for nuclei instance segmentation. The H&E-stained sections of 23 different human tissues were provided by Associate Professor Adolf Ellinger, PhD from the specimen collection of the Department of Cell Biology and Ultrastructural Research, Center for Anatomy and Cell Biology, Medical University of Vienna. We only obtained the stained tissue sections, not the original tissues. These images were only used for teaching purposes for a long time where no ethic votum applied. Some of the human tissues were formaldehyde-fixed, embedded in celloidin and sectioned at ≈\(15\mbox{--}20\,\mu m\) (jejunum, kidney, liver, oesophagus, palatine tonsil, pancreas, placenta, salivary gland, spleen, tongue). The other human tissues were formaldehyde-fixed and paraffin-embedded (FFPE) and sectioned at ≈\(4\mbox{--}5\,\mu m\) (cerebellum, cerebrum, colon, epiglottis, lung, melanoma, muscle, peritoneum, stomach (cardia), stomach (pylorus), testis, umbilical cord, and urinary bladder). Mouse tissue samples from bone (femur), fat (subscapularis), heart, kidney, liver, muscle (tibialis anterior muscle), spleen, and thymus were obtained from 8-week-old male C57BL/6 J mice28. 4 μm sections of the FFPE tissue samples were stained with H&E (ROTH, Austria) and coverslipped with Entellan (Merck, Germany). With one exception (human melanoma) all tissues in our dataset are healthy tissues.

Sample acquisition

WSIs were generated with a TissueFAXS (TissueGnostics, Austria) scanning system composed of an Axio Imager Z1 (Zeiss, Oberkochen, Germany), equipped with a Plan-Neofluar 40×/0.75 objective (40× air) in combination with the TissueFAXS Image Acquisition and Management Software (Version 6.0, TissueGnostics, Austria). Images were acquired at 8-bit resolution using a colour camera (Baumer HXG40c).

Field of view and patch selection

The scanning system stores individual 2048 × 2048 Field of Views (FOV) with their respective locations in order to be able to combine them into a WSI. Instead of using WSIs, we utilized the FOVs to generate the dataset. A senior cell biologist selected the most representative FOVs for each human and mouse WSI. From each FOV, a 512 × 512 pixel image was extracted by central cropping. These images were saved in lossless Portable Network Graphics (PNG) format. In total, 665 raw image patches were created to build the NuInsSeg dataset.

Generation of ground truth, auxiliary, and ambiguous area segmentation masks

We used ImageJ28 (version 1.53, National Institutes of Health, USA) to generate the ground truth segmentation masks. We followed the same procedure suggested by Mahbod et al.5 to label nuclei. We used the region of interest (ROI) manager tool (available on the Analysis tab) and the freehand option to delineate the nuclei borders. We manually draw the nuclei border for each instance until all nuclei were segmented for a given image patch. Although some semi-automatic tools such as AnnotatorJ with U-Net backbone29 could be used to speed up the annotation, we stuck to fully manual segmentation to prevent any hidden bias toward the semi-autonomic annotation method. The delineated ROIs were saved as a zip file, and the Matlab software (version 2020a) was then used to create binary and labeled segmentation images (as PNG files). Besides the original raw image patches, binary and labeled segmentation masks, we also publish a number of auxiliary segmentation masks that can be useful for developing computer-based segmentation models. Auxiliary segmentation masks, including border-removed binary masks, elucidation distance maps of nuclei, weighted binary masks (where higher weights are assigned in the borders of touching objects), are published along with our dataset. The developed codes to generate these masks are available on the published GitHub repository. Moreover, we annotated the ambiguous areas in all images of the dataset for the first time. Indicating ambiguous regions was partially provided in the test set of the MoNuSAC challenge30, but in this work, we provide it for the entire dataset. We used an identical procedure and software to create the ambiguous segmentation masks. These vague areas consist of image parts with very complex appearances where the accurate and reliable manual annotation is impossible. This is potentially helpful for in-depth analysis and evaluation of any automatic model for nuclei instance segmentation. Manual segmentation of nuclei and ambiguous areas detection were performed by three students with a background in cell biology. The annotations were then controlled by a senior cell biologist and corrected when necessary. Some example images, along with related segmentation and vague masks, are shown in Fig. 1.

Fig. 1
figure 1

Example images and manual segmentation masks of three human organs from the NuInsSeg dataset. The first three columns show the original images, the labeled and the binary mask, respectively. The represented images in the fourth to sixth columns show auxiliary segmentation masks that can be beneficial for the development of segmentation algorithms. The last column shows the vague areas where accurate and deterministic manual segmentation is impossible. Some images do not contain ambiguous regions, such as the represented spleen image in the last row.

Full size image

Data Records

The NuInsSeg dataset26 is publicly available on Zenodo (https://doi.org/10.5281/zenodo.10518968) and also on the Kaggle platform (https://www.kaggle.com/datasets/ipateam/nuinsseg). The related code to generate the binary, labeled, and auxiliary segmentation masks from the ImageJ ROI files is also available on the NuInsSeg published GitHub repository https://github.com/masih4/NuInsSeg. This dataset contains 665 image patches with 30,698 segmented nuclei from 31 human and mouse organs. The organ-specific details of the generated dataset are shown in Table 2. As shown in the table, the nuclei density in some tissues/organs (e.g., mouse spleen) is much higher in comparison to other tissues/organs (e.g., mouse muscle). This diversity allows for a more in-depth investigation into the capabilities of automatic models to handle different training and testing sizes for the nuclei instance segmentation task in the future studies.

Table 2 Details of the NuInsSeg dataset per human and mouse organ.
Full size table

Technical Validation

To create a baseline segmentation benchmark, we randomly split the dataset into five folds with an equal number of images per fold (i.e., 133 images per fold). We used the Scikit-learn Python package to create the folds with a fixed random state to reproduce the results (splitting code is available on the Kaggle and Github pages). Based on the created folds, we developed a number of DL-based segmentation models and evaluated their performance based on five-fold cross-validation. To facilitate to use of our dataset and developing segmentation models, we published our codes for two standard segmentation models, namely shallow U-Net and deep U-Net models31 on the Kaggle platform (https://www.kaggle.com/datasets/ipateam/nuinsseg/code?datasetId=1911713). The model architectures of the shallow U-Net and deep U-Net are very similar to the original U-Net model but we added drop out layers between all convolutional layers in both encoder and decoder parts. Four and five convolutional blocks were used in the encoder and decoder parts of the shallow U-Net and deep U-Net, respectively. The model architecture of these two models is publicly available at our published kernels on our NuInsSeg page on the Kaggle platform. Besides these two models, we also evaluated the performance of the attention U-Net32, residual attention U-Net32,33, two-stage U-Net34, and the dual decoder U-Net13 models. The architectural details of these models were published in the respective articles. We performed an identical five-fold cross-validation scheme in all experiments to compare the results. For evaluation, we utilized similarity Dice score, aggregate Jaccard index (AJI), and panoptic quality (PQ) scores as suggested in former studies5,6,35. The segmentation performance of the aforementioned models is reported in Table 3. As the results show, residual attention U-Net delivers the best overall Dice score between these models, but dual decoder U-Net provides the best average AJI and PQ scores. Interestingly, the dual decoder model achieved the best overall PQ score in the MoNuSAC post challenge leaderborad17,36, and it also achieved the best instance-based segmentation scores for the NuInsSeg dataset. It should be noted that these results can be potentially improved by using well-known strategies such as ensembling37, stain augmentation27,38 or test time augmentation39 but achieving the best segmentation scores is out of the focus of this study. Instead, these results could be used as baseline segmentation scores for comparison to other segmentation models in the future, given that the same five-fold cross-validation scheme is used.

Table 3 NuInsSeg segmentation benchmark results based on five-fold cross-validation.
Full size table

Usage Notes

Our dataset, including raw image patches, binary and labeled segmentation masks, and other auxiliary segmentation masks, is publicly available on the published NuInsSeg pages on Zenodo26 and Kaggle platform. Step-by-step instructions to perform manual annotations and related codes to generate the main and auxiliary segmentation masks are available at our published Github repository. We also provide three kernels on the Kaggle platform to facilitate using our dataset. One kernel is devoted to explanatory data analysis (EDA), where interested researchers can visualize and explore different statistics of the NuInsSeg dataset. The other two kernels consist of related codes to perform five-fold cross-validation based on two DL-based models, namely shallow U-Net and deep U-Net, as described in the previous section. Different Python packages were used in the coding of these kernels. To report statistics and visualize data in the EDA kernel, we mainly used Pandas (version 1.3.5) and Matplotlib (version 3.5.1) Python packages. For the DL-based model development, we mainly used Tensorflow (version 2.6.2), Keras (version 2.6.0) frameworks, and finally, for performing cross-validation, pre-and post-processing, and augmentation, Scikit-learn (version 0.23.2), Scikit-image (version 0.19.1) and Albumentation (version 1.1.0) were exploited, respectively.

In addition to publishing the dataset on a well-known repository (i.e., Zenodo26), we have also made it available on the Kaggle platform, which offers limited free computational resources. Therefore, interested researchers can directly access our dataset and develop ML- or DL-based algorithms to perform nuclei instance segmentation on the NuInsSeg dataset. However, there is no limitation to downloading and saving the dataset on local systems and performing analysis using local or other cloud-based computational resources.

It is worth mentioning that the NuInsSeg dataset can be used alone to train, validate, and test any segmentation algorithm, or it can be used as an independent test set to measure the generalization capability of already developed segmentation models.