Artificial intelligence by deep learning is a game changer in medical diagnostics, with results outperforming existing methods. We have previously developed a deep learning system that can predict colorectal cancer patient outcome from routine pathology images significantly better than current prognostic biomarkers. However, the predictions are not traceable by humans and this black box nature of the system is hampering clinical implementation.
The aim of the current project is to reveal the underlying biological mechanisms utilised by the deep learning system to obtain novel insight into carcinogenesis and to develop improved prognostic markers for cancer patients. Recent developments have provided methods that enhance our ability to visualise image areas of particular importance to deep learning predictions. From imaged tissue sections, we will measure a range of biological relevant factors in the same cells and tissue, and align with the regions that are important to the deep learning predictions. This approach is intended to identify combinations of biological factors and their context that is important for patient prognosis.
We will study prostate cancer and colorectal cancer in the project. Through this combined biological and machine learning approach, we intend to provide methods to make our own and similar networks more transparent and thus easier to use for clinicians, as well as to improve our understanding of the biological mechanisms underlying metastatic disease.
A rapidly increasing number of publications now demonstrate high performance of convolutional neural networks in medical diagnostics. However, few of these systems have reached the clinic, an important reason being their “black box” nature - the basis of their predictions is not traceable by humans. We have recently developed the DoMore-v1 classifier, a deep learning system for predicting patient outcome in colorectal cancer (Skrede et al., Lancet 2020;395:350-360). When independently tested on 1122 patients, the classifier outperformed all current prognostic biomarkers. However, an intriguing question remains: How can neural networks utilizing plain microscopic tissue images to predict patient outcome years later? Recent developments have provided methods that enhance our ability to visualize image areas of particular importance to network predictions. However, it seems unlikely that satisfactory understanding can be obtained without supplementing such information with concrete biomedical information, including biochemical measurements on cellular level. This latter information must be provided on image form, aligned to the images showing areas of particular importance to outcome predictions. Thus, the first work-packages in the project develop methods for simultaneous displaying various biochemical markers in pathological images and further develop tools for identifying the same cells in different images. The next packages aim at adapting and testing visualization methods to make them suitable for revealing features in pathological images utilized by prediction networks, while the last project activity is to connect important image characteristics and the biomedical markers.
Through this combined biological and machine learning approach, we intend to provide methods to make our own and similar networks more transparent and thus easier to use for clinicians, as well as to improve our understanding of the biological mechanisms underlying metastatic disease.