Development and visual assessment of a cross-modal Brain tumor segmentation pipeline

Abstract

Automatic segmentation is an important step in therapy planning for brain tumors, such as Vestibular Schwannoma. Treatment protocols include contrast-enhanced T1-weighted (ceT1) and high-resolution T2-weighted (hrT2) MR scans. Although ceT1 scans provide higher contrast, they use contrast agents which can cause cumulative side-effects. Therefore, efforts are underway to move to hrT2 completely. Because the availability of large, fully annotated data sets is limited, strategies for using cross-modality data are needed. After developing an automated algorithm, artificial intelligence (AI) engineers must evaluate the results of their models against ground truth labels and compare them to other algorithms. Visual assessment through Visual Analytics (VA) improves an in-depth understanding of such automated approaches. However, current VA applications are limited and do not provide flexible comparison capabilities that are able to drill down from large cohorts of patients into individual image slices. Also, they are not able to provide a view on correlations to other dataset- and image-derived features, such as from radiomics.This thesis has two main contributions. First, we develop two domain adaptation methods that transfer knowledge from ceT1 to hrT2 scans. The goal is to generate automatic tumor segmentation on hrT2 images. Cross-modal data of a cohort of 242 patients, each consisting of annotated ceT1 and non-annotated hrT2 scans, are used. The methods are enhanced with a classification-guided module which avoids false positive predictions of slices. Second, we design and implement an interactive web-based VA application for the assessment of algorithm performance and results. We perform a quantitative evaluation and demonstrate four use case scenarios. The proposed tool allows the users to compare multiple models and subjects on different levels of detail and find correlations between performance values and radiomics features. Our best methods achieve 61.14 % and 92.62 % Dice Score on only tumor slices and the entire dataset, respectively. Our VA approach provides additional insight, useful for the assessment of the developed algorithms.