Use cases

DATA

283 CTs, each presenting at least one lung affected:

• 19 pleurisy
• 126 caverns

Processed in a lung-wise manner -> 566 inputs:

• 444 affected
• 21 pleurisy
• 145 caverns.

1st APPROACH: Reducing volumetric data to 2D projections

1. Segmentation: identify the lungs and eliminate other parts

2. Projection: for each lung, for each dimension of the 3D matrix compute 3 statistics: mean, standard deviation, maximum

Result: a set of 3 RGB images for each lung, each channel corresponding to one of the 3 statistics

1st APPROACH: Feeding NNs with 2D projections

1. Data augmentation:

• horizontal and vertical flipping
• small degrees rotations
• blurring
• adding gaussian noise
• distortions
• random cropping
• changing different values of hue, saturation or brightness

2. Various NNs VGG, AlexNet, SqueezeNet, pretrained ResNet

2nd APPROACH: Exploiting whole volume

• 3D convolutions
• • achieving AUC < 0.60 on training set
• Slice fusion
• • achieving AUC ~0.60 on validation set

3rd (WINNING) APPROACH: Slice-wise classification

1. Manually identify the slices presenting affections in the training set, for each lung

2. Preprocess (crop the lungs after segmentation, change window level and range, augment)

3. Train a classifier at slice level (modified versions of InceptionNet and pretrained EfficientNet)

4. Aggregate the results per CT

Our Submissions

Future extensions

Exploit data from TBportal

• to improve our model
• to include other affections

Explore COVID databases

DATA

Ample multi-institutional routine clinically-acquired pre-operative multimodal MRI scans of glioblastoma and lower grade glioma, with pathologically confirmed diagnosis and available survival data; ground truth labels provided by expert board-certified neuroradiologists.

• 369 patients: 75 LGG + 261 HGG
• 5 files per patient: t1, t2, flair, t2-ce, mask

1st APPROACH: Segmentation

Model: 3D U-Net with Dense Connection

Training: Normalized Random Crops from the whole patient volume

2nd APPROACH: Segmentation

Model: 3D Cascaded UNet

Training: Trained on each label separately, the first model taking as input downgraded images and the second full size images for segmentation refining

3rd APPROACH: Segmentation

Model: U2Net

Training: takes as input normalized t1ce, t2 and flair modes as 3 channel images.

4th APPROACH: Segmentation

Model: Contour-Aware Segmentation Network with Encoder-Decoder Structure

Training: Trained to recognize the tumor and to reconstruct the original image

1st APPROACH: Overall survival

Model: ResNet18

Input: RGB image for each patient each channel corresponding to one of the 3 labels

2nd APPROACH: Overall survival

• t1, t2, flair, t2-ce are the input for a pretrained UNet model for brain segmentation
• With the features learned from the segmentation, the classifier predicts the type of survival (short, mid or long)

3rd APPROACH: Overall survival

Model: Radiomics and ExtraTrees for regression

Input: Radiomics extracted from segmentation for each patient + age

(Center of mass, Sphericity, SurfaceArea, Maximum3DDiameter, Flatness, %ET, %ET, NCT Existance..)

Unsupervised liver segmentation

1. Remove fat tissue by identifying thresholds on histograms

2. erode, dilate, binary_fill_holes and keep only the biggest connected graph extracted.

Supervised liver segmentation

Pre-processing: thresholding HUs, normalization

Training NNs:

• 3D RoI Aware Unet, 3D ResUNet
• Transfer Learning (MedicalNet)
• 2D U-Net with Convolutional LSTM (best approach)

Supervised tumour segmentation

Model: 2D U-Net with Convolutional LSTM for residual connections

Training: Thresholded picture of each slice for the specific interval for liver visualisation. Predicting with one single model, both liver and tumor location

Registration

The tumour is better detected in arterial time

Registration is needed to align the series and compare the acquisitions taken at different times

Next step: classification

COLON SEGMENTATION

Medical protocol suggests to first detect the colon and tumor in tha saggital plane, and then analyze the tumor in the axial plane.

Using Mask-RCNN, we conduct colon segmentation.

APPROACH

Model: 3D RoI-Aware neural network

Training: Multi-tasking learning, Contour Detection

Team-work: Constant feedback from doctors from IRO to improve the results

Registration

Comparing the results of a fixed pateint before and after non-operative treatment

For example, in this picture, in the left side the tumoral volume is 0.011 liters, whereas in 2020 the volume is 0.005.

Dataset

The dataset for the competition (https://www.drivendata.org/competitions/65/clog-loss-alzheimers-research/), is composed of more than half of million of videos with blood flow from mouse brain. There are only 600 videos with stalled vessels. The others are normal flowing vessels.

5th place on the leaderboard

Using three channels projections, we moved from video dataset to a 2D dataset in which the photos are very suggestive of the whole video. Stacking together the maximum, minimum and deviation projections we recreated the training set.

Best score solution

In order to take a high position on the leaderboard, we used the projection pictures randomly rotated at any degree. We also trained 7 different neural network to make an ensemble. At test time we predicted for each picture 11 label (stalled or not), each label being given based on the angle of the picture.