Visualization of deep learning in auto-delineation of cancer tumors

Abstract

Purpose: The deoxys framework, developed by Huynh and is available at https://github. com/huynhngoc/deoxys/, has the final goal of creating a user-friendly software that helps radiologists with tumor delineation problems. Currently, users of this framework can create and run any deep learning experiments using this framework. To increase the transparency and interpretability of the deep learning model, there is a need for adding model visualization methods into the deoxys framework. Therefore, in this thesis, several model visualization methods were implemented and integrated into the deoxys framework. In addition, this thesis also demonstrated the benefits of model visualization for users of the deoxys framework, including radiologists and data scientists.

Methods: Model visualization methods such as activation maps, activation maximization, saliency maps, deconvnet, and guided backpropagation were implemented in this Master’s thesis. The implementation of these visualization methods was assessed by comparing the results provided by the deoxys framework to previously published results.

The implemented visualization methods were applied to a deep learning model, which was trained on the head and neck cancer data of PET/CT scans for automatic tumor cancer delineation. The model visualization results were interpreted to demonstrate their benefits for model understanding.

Results: The implementation of the model visualization methods reproduced similar results with previous studies, thus passed the quality control assessment and ensured the reliability of the implemented visualization methods.

When interpreting the visualization results, the pretrained model was found to extract the tongues, bones, muscles and glands from the CT scans and the lymph nodes from the PET scans. In addition, the pretrained model had a high chance of marking a pixel as cancer tumors if that pixel belonged to the bright lymph nodes in the PET scan. Moreover, the weakness of the pretrained model such as the lack of data augmentation was found during the interpretation.

Conclusions: The model visualization methods were demonstrated to benefits both radiologists and data scientists. Radiologists can have internal insights of the deep learning model, while data scientists can find existing problems to improve the deep learning model performance.

Despite the existing limitations, the developed deoxys framework has the potential for improvement. This includes enhancement of the implemented visualization methods, and the addition of other model visualization methods. Ideally, an interactive user interface should be developed to satisfy the user-friendly goal of the deoxys framework.