Multistain Immnumohistochemistry Tissue Segmentation using End-to-end Color Deconvolution Deep Neural networks
DATASET
APPROACHES
UNET multiple stainings
CD-UNET
UNET single staining
METHODS
COLOR DECONVOLUTION segment
UNET
Original UNET
Groundtruth: 1 per tile with: pathologists annotations (512x512x1 )
Class Imbalance: inherent imbalance of classes : use loss weghting strategy = median frequency baslancing
Each slide: split into
overlapping 512x512x3
at 10x (half or.resolution)
Data Augmentation: hue, brightness, scale jittering, add uniform noise to intensity channel, image flips, small random affine transformations
Annotations:
51 slides training (10% for validation) 26 slides testing
77 WSI colorectal carcinoma. H&E+8 IHC = 9 stains
Tissue: 16% (pixels)
Tumor: 19%
Necrosis: 7%
Exclude= irrelevance, artifacts,etc
Bacground:58%
Lower resolution needed for increasing contextual information when classifying pixel info
Optimization: SGD optimizer with momentum of 0.9
Multiple GPUs with synchronous SGD
Loss function: average of cross entropy loss. loss for ignored pixels=0 weight
GPUs: 15
Output: 512x512x4 with probabilities for each class for each pixel
Batch size:240 (16 per GPU)
Input: RGB 512x512 to range [0 1]
Default learning rate per GPU=0.001
Distributed module: PyTorch
UNET training
Dropout layers at the endo of the encoder and the decoder (less overfitting)
Batch normalization after every convolutional layer
Smaller network width: half the number of filters per laayer (higher speed and less overfitting)
Zero padding in all convulutional layers (preserve input size)
Increased generalization capabilities
Smooth convergence of F1 scores after 200 epochs
No smooth convergence.
Big difference for necrosis and background
Prefers to train a single model for all stains rather than multiple models
3 filters in second layer to mantain architecture's input size
Each layer followed by nonlinear function
6 filters in the first layers for each stain color
ReLu and Batch Normalization
Color deconvolution parameters learned as part of segmentation network
No predefinition of stain basis parameters
Faster learning
Better generalization
F1 scores converge faster and smoother. Higher Scores
RESULTS
Color Deconvolution Segment Visualization
Apply activation maximization on filters of first layer
CD segment output: visualize output with different stains
Activation max of the first layer filters: generate synthetic images that max activate response
Feature visualization
Insert noise and select target output
Synthetic images for tissue: regular cell structures in the center and far from it
Max tumor and necrosis score: look for patterns of condensed large distorted cell nuclei around target pixel
Max tissue score: tissue cell nuclei far from target pixel. patterns of multiple tissue cell nuclei around target pixel
Pixel Attribution
Process: use smooth grad + average score gradients → ReLu → threshold gradient image
Tumor: the gradients higlight tumor cell nuclei in target pixel surrounding and ignore other textures
Effective receptive field different between categories
Normal tissue : the gradients higlight healthy nuclei in target pixel surrounding and ignore other textures. Large effective receptive field