Allocation of clinical resources in oncology depends on a physician’s ability to predict recurrence and survival events and rates. Currently, clinicians use a tumor’s stage, grade and type as primary determinants of recurrence. Over the last decade, dozens of predictive models have been developed to improve on the ability of stage, grade and type to predict recurrence. Many are used in every day practice to develop plan surveillance strategies, administer adjuvant therapy and develop eligibility for clinical trials. Unfortunately, many of the prediction models used by clinicians to guide patient care and clinical trial selection for solid tumors may not be as accurate as they are thought to be, according to a new study by researchers at Fox Chase Cancer Center.
“Medicine has come to rely on prognostic models and has put a lot of time and resources into developing them, but they are far less robust than we would have hoped,” said study author Robert G. Uzzo, MD, FACS, Chair of the Department of Surgical Oncology at Fox Chase Cancer Center. “In some cases, using these models to predict future events is not much better than the flip of a coin.”
Uzzo and colleagues recently conducted a study to validate some of the currently accepted models used to predict whether patients with renal cell carcinoma (RCC) will have their disease return after treatment.
Historically, clinicians relied primarily on a combination of stage, grade, and histology to predict oncologic events – known as the TNM system. More recently, Uzzo explained, several institutions have used retrospective data to develop prognostic models that combine clinical and/or pathologic variables to help predict how long a patient may stay in remission (recurrence-free survival), or how long they might survive (overall survival).
“If a patient has just had surgery to have a cancer removed, common questions from the patient are, ‘Doctor, will this cancer come back?’ or ‘Is this going to kill me?’” Uzzo said. “Clinicians rely on these big data models to communicate these risks to the patient and help guide decisions on whether or not the patient should get additional therapy, or be enrolled in a clinical trial.”
However, there are a couple of weaknesses regarding these models: they were developed prior to the existence of many of our successful systemic therapies for metastatic RCC, and they use retrospective data.
In this study, Uzzo and colleagues tested the accuracy of the eight most commonly used RCC models using prospective data from the large adjuvant kidney cancer trial known as the ASSURE trial. The use of prospective data instead of retrospective data allowed for a centralized validation of the clinical and pathological variables inputted into the models, and a standardized reporting of the outcomes measured.
The paper, “Predicting renal cancer recurrence: Defining limitations of existing prognostic models with prospective trial based validation” was published in the Journal of Clinical Oncology.
Accuracy was measured using C-index, a measure of how well the model predicts what the researchers thought it would predict. If the C-index is 0.5, it is equivalent to the flip of a coin; if it is 1.0, it has a perfect correlation, Uzzo explained.
Among the eight models tested, the best performing (the SSIGN model) had a c-index of 0.688. The worst performing model (UISS) had a c-index of 0.556. Most of the tested models only marginally outperformed the more traditional TNM staging system (c-index 0.60).
“In the initial studies of these models, or in their subsequent validation studies, most had a predictive index somewhere around 0.8 or 0.9, but our study showed indexes closer to 0.6, which is not that much more predictive than TNM,” Uzzo said.
Although this study examined RCC-specific models, these issues may not be unique to this disease, and should be examined in other cancers using predictive models, such as breast, bladder, or prostate cancer, he said.
“Right now, these models are all we have to guide treatment or help decide on enrollment in a clinical trial,” Uzzo said. “But this data shows that we should not rely only on these models and that they should be more rigorously tested.”
This work was supported by the National Cancer Institute of the National Institutes of Health under the following award numbers: CA180820, CA180794, CA180867, CA180858, CA180888, CA180863, and Canadian Cancer Society #704970.