Keywords

benchmark dataset, pathology foundation models, Gleason grading, WSI-specific feature collapse, model robustness, prostate cancer

Abstract

Artificial intelligence foundation models are increasingly deployed for prostate cancer Gleason grading, where GP3/GP4 distinction directly impacts treatment decisions (active surveillance vs. intervention). However, these models may achieve high validation accuracy by learning specimen-specific artifacts rather than generalizable biological features, limiting real-world clinical utility. We introduce PANDA-PLUS-Bench, a curated benchmark dataset derived from expertly annotated prostate biopsies designed specifically to quantify this failure mode. The benchmark comprises nine carefully selected whole slide images from nine unique patients containing diverse Gleason patterns, with non-overlapping tissue patches extracted at both 512 × 512 and 224 × 224-pixel resolutions across eight augmentation conditions. Using this benchmark, we evaluate seven foundation models (Virchow, Virchow2, UNI, UNI2, Phikon, Phikon-v2, and HistoEncoder) on their ability to separate biological signals from slide-level confounders. Our results reveal substantial variation in robustness across models: the Virchow models achieved the lowest slide-level encoding among large-scale models (slide ID accuracy: 80.7–81.0%), yet Virchow2 exhibited the lowest cross-slide accuracy (47.2%). HistoEncoder, trained specifically on prostate tissue, demonstrated the highest cross-slide accuracy (59.7%) and the strongest slide-level encoding (slide ID accuracy: 90.3%), suggesting tissue-specific training may enhance both biological feature capture and slide-specific signatures. All models exhibited measurable within-slide vs. cross-slide accuracy gaps, though the magnitude varied from 19.9 percentage points (HistoEncoder) to 26.9 percentage points (Phikon). We provide an open-source Google Colab notebook enabling researchers to evaluate additional foundation models against our benchmark using standardized metrics. PANDA-PLUS-Bench addresses a critical gap in foundation model evaluation by providing a purpose-built resource for robustness assessment in the clinically important context of Gleason grading.

Original Publication Citation

Ebbert, J. L., & Della Corte, D. (2026). PANDA-PLUS-Bench: A Clinical Benchmark for Evaluating the Robustness of AI Foundation Models in Prostate Cancer Diagnosis. AI in Medicine, 1(2), 14. https://doi.org/10.3390/aimed1020014

Document Type

Peer-Reviewed Article

Publication Date

2026-05-28

Publisher

MDPI

Language

English

College

Computational, Mathematical and Physical Sciences

Department

Physics and Astronomy

University Standing at Time of Publication

Associate Professor

Download

Share

COinS