Chemotherapy success calculated by new machine learning model

A machine learning model has been created and trained by researchers at Johns Hopkins Medicine to calculate percent necrosis (PN), what percentage of a tumour is “dead” and no longer active, in patients with osteosarcoma, a type of bone cancer. When compared to the results of a musculoskeletal pathologist, the model’s calculation was 85 percent correct, and upon removing one outlier, the accuracy rose to 99 percent.

A post-chemotherapy PN calculation helps provide the patient with a prognosis for survival. For example, a PN of 99 percent indicates that 99 percent of the tumour is dead, showing that chemotherapy was effective, and the patient has an improved chance of surviving. To calculate PN, pathologists look at, interpret, and annotate whole-slide images (WSIs), thinly sliced sections of a specimen that are mounted onto slides for microscopic analysis.

Dr Christa LiBrizzi, co-first author of the study and resident with Johns Hopkins Medicine’s Department of Orthopaedic Surgery, said: “Calculating the PN is a labour-intensive process that requires a lot of annotation data from the musculoskeletal pathologist.” She continued: “Additionally, it has low interobserver reliability, meaning that two pathologists trying to calculate a PN from the same WSIs will often report different conclusions. Due to these factors, we thought trying to calculate a PN by alternate means was a worthwhile effort.”

The team aimed to develop a “weakly supervised” machine learning model that required minimal annotation data to be trained on. This would mean that musculoskeletal pathologists using the model to calculate a patient’s PN would only need to provide it with partially annotated WSIs and therefore reduce the pathologist’s labour burden.

To begin, the scientists gathered data, including WSIs, from the pathology archives of Johns Hopkins’ U.S. tertiary cancer centre. All data came from patients with osteosarcoma that originated in the centre of the bone, known as intramedullary osteosarcoma, who underwent chemotherapy and surgery at the centre between 2011 and 2021. A musculoskeletal pathologist partially annotated three types of tissue on each of the gathered WSIs: active tumour, dead tumour, and non-tumour tissue, and estimated the PN for each patient. The team then trained the model using this information.

Zhenzhen Wang, co-first author of the study and a doctoral student in Biomedical Engineering at the Johns Hopkins University School of Medicine explained: “We decided to train the model by teaching it to recognise image patterns.”  She continued: “We segregated the WSIs into thousands of small patches, then divided the patches into groups based on how they were labelled by the pathologist. Finally, we fed these grouped patches into the model to train it. We thought this would give the model a more robust frame of reference than simply feeding it one large WSI and risking missing the forest for the trees.”

After being trained, the model and the musculoskeletal pathologist were given six WSIs to interpret from two osteosarcoma patients. The results demonstrated an 85 percent positive correlation between the model and the pathologist’s PN calculations and tissue labelling. However, the model did not always properly label cartilage, which resulted in one outlier because of an abundance of cartilage on one WSI. When the outlier was removed, the correlation increased to 99 percent.

“If this model were to be validated and produced, it could help expediate the evaluation of chemotherapy’s effectiveness on a patient — and thus, get them a prognosis estimate sooner,” noted LiBrizzi. “That would reduce health care costs, as well as labour burdens on musculoskeletal pathologists.”

The teams’ future goal is to include cartilage tissue in the model’s training and to diversify the WSIs to include other types of osteosarcoma beyond intramedullary.

The study was published in the Journal of Orthopaedic Research.