An annotated fluorescence image dataset for training nuclear segmentation methods.

Patient samples Tumor ( documentclass[12pt]{minimal} usepackage{amsmath} usepackage{wasysym} usepackage{amsfonts} usepackage{amssymb} usepackage{amsbsy} usepackage{mathrsfs} usepackage{upgreek} setlength{oddsidemargin}{-69pt} begin{document}$$n$$end{document} n = 4) and bone marrow ( documentclass[12pt]{minimal} usepackage{amsmath} usepackage{wasysym} usepackage{amsfonts} usepackage{amssymb} usepackage{amsbsy} usepackage{mathrsfs} usepackage{upgreek} setlength{oddsidemargin}{-69pt} begin{document}$$n$$end{document} n = 4) samples of stage M neuroblastoma patients, the Schwann cell stroma-rich part of a patient with a ganglioneuroblastoma tumor and one wilms tumor patient were obtained from the Children’s Cancer Research Institute (CCRI) biobank (EK.1853/2016) within the scope of ongoing research projects. In addition, two patient-derived neuroblastoma cell lines (CLB-Ma, STA-NB10) were used. Written informed consent has been obtained from patients or patient representatives. Ethical approval for IF staining and imaging was obtained from the ethics commission of the Medical University of Vienna (EK1216/2018). All authors confirm that we have complied with all relevant ethical regulations.

Preparation and IF-staining of tumor tissue cryosections

The fresh-frozen tumor tissues of one ganglioneuroblastoma patient, one neuroblastoma patient and one Wilms tumor patient were embedded into tissue-tek-OCT and 4 documentclass[12pt]{minimal} usepackage{amsmath} usepackage{wasysym} usepackage{amsfonts} usepackage{amssymb} usepackage{amsbsy} usepackage{mathrsfs} usepackage{upgreek} setlength{oddsidemargin}{-69pt} begin{document}$$mu m$$end{document} μ m thick cryosections were prepared. Sections were mounted on Histobond glass slides (Marienfeld), fixed in 4.5% formaledhyde and stained with 4,6-diamino-2-phenylindole (DAPI), a blue fluorescent dye conventionally used for staining of nuclei for cellular imaging techniques. Finally, slides were covered with Vectashield and coverslips were sealed on the slides with rubber cement. Preparation and IF-staining of HaCaT human skin keratinocyte cell line The HaCaT cell line, a spontaneously transformed human epithelial cell line from adult skin 20 , was cultivated either in culture flasks or on microscopy glass slides. Cell cultures were irradiated (2 and 6 Gy), harvested, cytospinned, air-dried and IF stained. Cells grown and irradiated on the glass slides were directly subjected to IF staining. Cells were fixed in 4% formaldehyde for 10 minutes at 4 °C, and were permeabilized with 0.1% sodium dodecyl sulfate (SDS) in PBS for 6 minutes. Slides were mounted with antifade solution Vectashield containing DAPI and coverslips were sealed on the slides with rubber cement. Preparation and IF-staining of tumor touch imprints and bone marrow cytospin preparations Touch imprints were prepared from fresh primary tumors of 4 stage M neuroblastoma patients as previously described 21 . Mononuclear cells were isolated from bone marrow aspirates of 3 stage M neuroblastoma patients by density gradient centrifugation and cytospinned as described 22 . After fixation in 3.7% formaldehyde for 3 minutes, cells were treated according to the Telomere PNA FISH Kit Cy3 protocol (Dako), mounted with Vectashield containing DAPI, covered and sealed.

Preparation and IF-staining of neuroblastoma cell line cytospin preparations

STA-NB-10 and CLB-Ma are cell lines derived from neuroblastoma tumor tissue of patients with stage M disease. Preparation and drug-treatment were conducted as described 23 . Briefly, cells were cultured in the absence or presence of 5 nM topotecan, a chemotherapeutic drug for 3 weeks, detached and cytospinned to microscopy glass slides. Preparations were air-dried, fixed in 3.7% formaldehyde, immuno- and DAPI stained, covered and sealed.

Fluorescence imaging

Samples were imaged using 1. an Axioplan-II microscope from Zeiss equipped with a Maerzhaeuser slide scanning stage and a Metasystems Coolcube 1 camera using the Metafer software system (V3.8.6) from Metasystems, 2. an Axioplan-II microscope from Zeiss equipped with a Zeiss AxioCam Mrm 1 using the Metasystems ISIS Software for microscopy image acquisition, 3. an LSM 780 microscope from Zeiss equipped with an Argonlaser 458 nm, a photomultiplier tube (PMT) detector (371–740 nm) and a motorized Piezo Z-stage using the Zeiss Zen software package and 4. a SP8X from Leica equipped with a Diode Laser and a PMT detector (447–468 nm). For the presented dataset, we digitized the DAPI staining pattern representing nuclear DNA. Additional immunofluorescence or FISH stainings were in part available. An automatic illumination time was set as measured by pixel saturation (Metasystems Metafer and ISIS) or defined manually (Zeiss and Leica LSMs). Objectives used were a Zeiss Plan-Apochromat 10 documentclass[12pt]{minimal} usepackage{amsmath} usepackage{wasysym} usepackage{amsfonts} usepackage{amssymb} usepackage{amsbsy} usepackage{mathrsfs} usepackage{upgreek} setlength{oddsidemargin}{-69pt} begin{document}$$times $$end{document} × objective (Zeiss Axioplan II; numerical aperture 0.45; air), a Zeiss 20 documentclass[12pt]{minimal} usepackage{amsmath} usepackage{wasysym} usepackage{amsfonts} usepackage{amssymb} usepackage{amsbsy} usepackage{mathrsfs} usepackage{upgreek} setlength{oddsidemargin}{-69pt} begin{document}$$times $$end{document} × Plan-Apochromat (Zeiss LSM 780; numerical aperture 0.8; oil), a Zeiss 20 documentclass[12pt]{minimal} usepackage{amsmath} usepackage{wasysym} usepackage{amsfonts} usepackage{amssymb} usepackage{amsbsy} usepackage{mathrsfs} usepackage{upgreek} setlength{oddsidemargin}{-69pt} begin{document}$$times $$end{document} × Plan-Apochromat (Leica SP8X; nuermical aperture 0.75; oil), a Zeiss Plan-Neofluar 40 documentclass[12pt]{minimal} usepackage{amsmath} usepackage{wasysym} usepackage{amsfonts} usepackage{amssymb} usepackage{amsbsy} usepackage{mathrsfs} usepackage{upgreek} setlength{oddsidemargin}{-69pt} begin{document}$$times $$end{document} × objective (Zeiss Axioplan II; numerical aperture 0.75) and a Zeiss Plan-Apochromat 63 documentclass[12pt]{minimal} usepackage{amsmath} usepackage{wasysym} usepackage{amsfonts} usepackage{amssymb} usepackage{amsbsy} usepackage{mathrsfs} usepackage{upgreek} setlength{oddsidemargin}{-69pt} begin{document}$$times $$end{document} × objective (Zeiss Axioplan II, Zeiss LSM 780 and Leica SP8X; numerical aperture 1.4; oil). Representative field of views (FOVs) were selected according to the following quality criteria: sharpness, intact nuclei and a sufficient number of nuclei.

Ground truth annotation

Nuclei image annotation was performed by students and expert biologists trained by a disease expert. To accelerate the time consuming process of image annotation, a machine learning-based framework (MLF) was utilized supporting the process of annotation by learning characteristics of annotation in multiple steps 24 . The MLF annotations result in a coarse annotation of nuclear contours and have to be refined to serve as ground truth annotation. Therefore, annotated images were exported as support vector graphic (SVG) files and imported into Adobe Illustrator (AI) CS6. AI enables the visualization of annotated nuclei as polygons overlaid on the raw nuclear image and provides tools to refine the contours of each nucleus. An expert biologist and disease expert carefully curated all images by refining polygonal contours and by removing polygons or adding them, if missing. Finally, an expert pathologist was consulted to revise all image annotations and annotations were curated according to the pathologist’s suggestions. In cases where decision finding was difficult, a majority vote including all experts’ suggestions was considered and annotations were corrected accordingly. Images were exported and converted to Tagged Image File Format (TIFF) files, serving as nuclear masks in the ground truth dataset. A sample workflow is illustrated in Fig. 1 . Fig. 1 Workflow for ground truth image annotation. ( a ) Raw image visualizing HaCaT cytospinned nuclei. ( b ) A machine learning framework was used to annotate the raw image, learning from user interaction within three consecutive steps: S1. foreground extraction, S2. connected component classification (red = non-usable objects, blue = nuclei aggregations, green = single nuclei) and S3. splitting of aggregated objects into single nuclei, resulting in an annotation mask. ( c ) Zoom-in of the SVG-file showing the nuclear image overlaid with polygons representing each annotated nucleus. Polygons were modified by expert biologists to fit effective nuclear borders. Challenging decisions on how to annotate nuclei, mainly occurring due to aggregated or overlapped nuclei, were presented to an expert pathologist and corrected to obtain the final ground truth. ( d ) The curated SVG-file was transformed into a labeled nuclear mask. Dataset split As the dataset is intended to be used to train and evaluate machine learning-based image segmentation methods, we created a dataset split into training set and test set. The training set consists of multiple images of ganglioneuroblastoma tissue sections, normal cells (HaCaT) as cytospin preparations or grown on slide, and neuroblastoma tumor touch imprints and bone marrow preparations. For each of these types of preparation, multiple images using the same magnification (20x or 63x) imaged with the same modality (Zeiss Axioplan II and the Metasystems Metafer Software) and showing a good signal-to-noise ratio were included. The test set consists of additional images of these preparation types and moreover, includes images of different preparation types (e.g. neuroblastoma cell line preparations, Wilms tumor and neuroblastoma tumor tissue sections) imaged with different modalities (Zeiss and Leica confocal LSM; Zeiss and Metafer ISIS software) and different signal-to-noise ratios. To enable an objective comparison of image segmentation architectures to the ground truth annotations with respect to varying imaging conditions, we classified each image of the test set into one of 10 classes, according to criteria such as sample preparation, diagnosis, modality and signal-to-noise ratio. The details are presented in Table 1 . The recommended dataset split into training set and test set and the test set classes can be downloaded along with the dataset. Table 1 Test set split into 10 classes to evaluate the generalizability of machine learning-based image segmentation methods with respect to varying imaging conditions. Acronym Description GNB-I ganglioneuroblastoma tissue sections GNB-II ganglioneuroblastoma tissue sections with a low signal-to-noise ratio NB-I neuroblastoma bone marrow cytospin preparations NB-II neuroblastoma cell line preparations imaged with different magnifications NB-III neuroblastoma cell line preparations imaged with LSM modalities NB-IV neuroblastoma tumor touch imprints NC-I normal cells cytospin preparations NC-II normal cells cytospin preparations with low signal-to-noise ratio NC-III normal cells grown on slide TS other tissue sections (neuroblastoma, Wilms) Single-nuclei annotation To set a baseline for machine learning-based image segmentation methods and to validate the proposed dataset, 25 nuclei were randomly sampled from the ground truth annotations for each of the classes, marked on the raw images and presented to two independent experts for image annotation. Annotation was carried out by a biology expert with long-standing experience in nuclear image annotation, further called annotation expert, and a biologist with experience in cell morphology and microscopy, further called expert biologist. Nuclei were annotated using SVG-files and Adobe illustrator. The single-nuclei annotations, described as single-cell annotations within the dataset, can be downloaded along with the dataset.

Annotation criteria

The annotation of nuclei in tissue sections or tumor touch imprints is challenging and may not be unambiguous due to out-of-focus light or nuclei, damaged nuclei or nuclei presenting with modified morphology due to the slide preparation procedure. We defined the following criteria to annotate nuclear images: Only intact nuclei are annotated, even if the nuclear intensity is low in comparison to all other nuclei present. Nuclei have to be in focus. If parts of a nucleus are out of focus, only the part of the nucleus being in focus is annotated. Nuclear borders have to be annotated as exact as resolution and blurring allows for. Nuclei are not annotated if their morphology was heavily changed due to the preparation procedure. Nuclei from dividing cells are annotated as one nucleus unless clear borders can be distinguished between the resulting new nuclei.