Head and neck cancer treatment outcome prediction: a dynamical low-rank model approach

Abstract

Machine learning (ML) has emerged as an essential tool for optimizing processes across diverse sectors, including healthcare. However, ML models for cancer detection, diagnosis, and research often require Deep Neural Networks (DNNs), which typically impose heavy computational demands and significant memory usage. Simultaneously, society is growing concerned about the extensive use of ML models, particularly those that are heavily energy-intensive, such as DNNs. Therefore, developing and facilitating energy-efficient and less resource-intensive DNNs is necessary.

This thesis investigated a promising alternative, Dynamical Low-Rank Approximation (DLRA), assessing it as a competitive technique for creating energy-efficient and less resource-intensive DNNs. This research showcased the potential of DLRA, using tabular healthcare data of patients with head and neck cancer (HNC) from Oslo University Hospital (OUS) and Maastricht University Medical Center (MAASTRO). This thesis aimed to evaluate how low-rank approximated neural networks (NN) compared to traditional dense neural networks in predicting cancer treatment outcomes, assessing their performance, energy consumption, and economic impact.

The DLRA models developed in this thesis were constructed with PyTorch, using two neural networks as foundational references. The analysis assessed the models' performance and generalizability through various methods, incorporating eight performance metrics to provide a comprehensive overview of their outcomes. The energy consumption for model inference was estimated by converting calculated FLOPs into kWh and further compared with those of OpenAI’s GPT-3 model. Additionally, a cost and sensitivity analysis were conducted, utilizing estimated energy consumption and the average Norwegian power price for 2022-2023 to illustrate the potential economic and energy impacts of DLRA.

The findings of this thesis demonstrate that an adaptive DLRA NN can outperform a dense NN in predicting HNC treatment outcomes, despite a compression rate of 43 \% for model inference. Additionally, this thesis shows that DLRAs relevance, in terms of potential energy and cost savings, solely, is not large for this thesis' models, but likely is for larger models like GPT-3. Furthermore, this thesis underscores the need for tools and routines that simplify the measurement and reporting of energy consumption in machine learning models. Advancing research in this area is crucial for developing and adopting more sustainable and effective machine-learning models.