UnMICST: Deep learning with real augmentation for robust segmentation of highly multiplexed images of human tissues

Abstract

Newly developed technologies have made it feasible to routinely collect highly multiplexed (20-60 channel) images at subcellular resolution from human tissues for research and diagnostic purposes. Extracting single cell data from such images requires efficient and accurate image segmentation. This starts with identification of nuclei, a challenging problem in tissue imaging that has recently benefited from deep learning. In this paper we demonstrate two generally applicable approaches to improving segmentation accuracy as scored using new human-labelled segmentation masks spanning multiple human tissues. The first approach involves the use of “real augmentations” during training. These comprise defocused and saturated image data and improve model accuracy when computational augmentation (Gaussian blurring) does not. The second involves collection of nuclear envelope data. The two approaches cumulatively and substantially improve segmentation with three different deep learning frameworks, yielding a set of high accuracy segmentation models. Moreover, the use of real augmentation may have applications outside of microscopy.