Quick and Accurate Tumor Follow-up for Clinical Routine

Monitoring Tumor Therapy More Precisely

We accelerate routine tumor follow-up and enhance reliability through software-assisted workflows.
© Fraunhofer MEVIS
Artificial intelligence helps monitor changes during the course of a tumor therapy.

Cancer patients generally need to attend regular checkups, which help determine how the patient responds to a new treatment.  The Fraunhofer Institute for Digital Medicine MEVIS is working on a method supported by AI to carry out follow-up checkups more quickly and accurately.


A patient has been receiving chemotherapy for some time. Now his doctor wants to know how well the drug has performed. Did the tumor shrink as hoped? Would it be better to switch to another medication? To determine this, a CT scan is typically made after three months and compared with the image from the onset of treatment. Until now, this has been performed with the naked eye supplemented by fairly rudimentary electronic aids that measure tumor diameter. Fraunhofer MEVIS is developing an automated procedure to achieve before-and-after comparisons, thus saving time and producing more accurate findings.

Today, comparing old and new pictures is rather complex. To avoid missing any newly formed metastases, specialists have to sift through three-dimensional CT data sets slice by slice - a time-consuming process. Another shortcoming, according to MEVIS researcher Jan Hendrik Moltz, is that “it is relatively imprecise to determine only the diameter of a tumor. Tumors and metastases are three-dimensional objects. A change in diameter does not necessarily reflect the relevant change in size: how the volume of the tumor has transformed as result of the therapy.”


A better way to compare pictures

The team is working on an automated solution to evaluate the therapeutic response with more speed and accuracy. Researchers want to use a method called registration to facilitate image comparison. The computer can superimpose identical image sections in two different images for direct comparison: Clicking on a certain point in the new image automatically marks the corresponding position in the old image.

This lets personnel more easily detect whether metastases have vanished, or new ones have formed. The system will automatically visualize differences in the images, for example, by marking or highlighting them. “Fraunhofer MEVIS is in a great position when it comes to registration,” says Moltz. “Radiologists are often amazed at how well it works.”

The second step, measuring tumor volume precisely and automatically, is more demanding. For this, accurate segmentation is required. The computer should be able to recognize what is and isn’t a tumor in the CT image. Such methods have been available for some time, but until now have usually required human intervention, limiting their practical application in hospitals and doctor’s practices. “Deep learning technologies should bring critical progress in this area,” says Moltz. “We hope to prepare the system for the clinical routine.”


An AI that recognizes tumors

To do so, the experts train a segmentation algorithm using extensive CT data sets. This contains detailed information about the appearance of tumors and metastases on the image and how they vary from surrounding tissue. Moltz and his colleagues also hope to integrate knowledge gained from previous work. “We are very experienced in segmentation,” says the computer scientist. “This sets us apart from groups that are only now beginning and employ purely data-based approaches.”

Initially, the Fraunhofer team wants to concentrate on the lung and liver, two organs that are especially relevant in oncology. Over the long term, however, the algorithm should be able to identify metastases throughout the body reliably and measure them precisely. The research group will also try to discover parameters from the segmentation in addition to tumor volume. These could include, for example, clues about tumor tissue composition and help describe changes in the tumor, thus providing supplementary information about the therapy response.

An initial prototype to demonstrate the registration is likely to be completed in a year. In five years, the software is expected to provide automatic tumor segmentation for organs such as the liver and lungs.


Two CT images in the course of therapy with subtraction representation indicating a newly formed bone tumor
© Fraunhofer MEVIS
A pair of follow-up CT scans with the difference image showing a new metastasis in the spine.

Even the smallest changes in CT scans can be crucial for choosing the best therapy. Registration establishes anatomical correspondences and enables the computation of difference images where new metastases can be better seen.

Change Visualization

Volume rendering of a lung showing the volume development of tumor volume.
© Fraunhofer MEVIS
Three-dimensional visualization of multiple lung metastases with color-coded change in size.

During chemotherapy, the change in metastasis size is examined with regular CT scans. A visualization can help the treating physicians and patients assess the therapeutic success.


CT image, table with clinical parameters and heatmap showing their correlation
© Fraunhofer MEVIS
Radiomics discovers relationships between image features and clinical data. The “heatmap” groups patients with similar features and represents them in areas of the same color.

Radiomics uncovers relationships between image features and clinical data. We apply techniques from statistics and machine learning to predict patient outcomes such as survival, therapy response, and side effects.