# Validation Is Not Enough: A Duplicate-Aware, Reproducible Frozen-Feature Benchmark for Brain Tumour MRI Classification on BRISC 2025

**Authors:** Narasimha Rao Vajjhala (Department of Computer Science, American University in Bulgaria, Blagoevgrad, Bulgaria) — ORCID 0000-0002-8260-2392; Ervin Ramollari (Department of Computer Science, University of New York Tirana, Tirana, Albania) — ORCID 0009-0006-8614-7813; Rajesh Kumar (Department of Computer Science, University of New York Tirana, Tirana, Albania) — ORCID 0000-0001-8223-828X
**Type:** Journal article (open access)
**Source:** Information, 17(11), Article 1175
**Published:** 2026-11-19
**DOI:** https://doi.org/10.3390/info17111175
**Canonical page:** https://www.narasimharao.net/research/duplicate-aware-frozen-feature-benchmark-brain-tumour-mri-brisc-2025/
**Indexing:** Scopus Q1 (CiteScore 8.2) · Web of Science ESCI Q2 · Impact Factor 4.3 (JCR 2025)
**Keywords:** brain tumour MRI classification; benchmarking; reproducibility; data leakage; near-duplicate detection; foundation models; DINOv2; linear probing; calibration; selective prediction

## Summary

Vajjhala et al. (2026) audited the BRISC 2025 brain tumour MRI dataset for near-duplicate images with a perceptual hash and benchmarked frozen ResNet-18, ResNet-50 and DINOv2 ViT-S/14 features under an identical linear probe on a duplicate-aware split of 5,477 images. The audit found 1,352 candidate near-duplicate pairs, 439 of them crossing the official train/test split, and 104 training images pixel-identical to test images. DINOv2 was numerically ahead (macro-F1 77.3%, macro-AUROC 93.2%), but no pairwise difference survived Holm correction, and in-pool validation overstated official test macro-F1 by 9.7–13.5 points.

**Practical implication.** Researchers using public MRI slice collections without patient identifiers should audit and document duplicates before splitting, the authors argue, because the partition deserves as much scrutiny as the model.

- **The Problem:** Public brain tumour MRI collections are routinely reported with near-perfect classification accuracy, yet they carry no patient identifiers and their train/test overlap is rarely quantified.
- **The Methodology:** Dataset audit and controlled benchmark: a perceptual-hash near-duplicate audit of the BRISC 2025 brain tumour MRI dataset (6,000 slices), a duplicate-aware group-aware split of 5,477 images, and a linear probe on frozen ResNet-18, ResNet-50 and DINOv2 ViT-S/14 features over three seeds, with group-level bootstrap confidence intervals, paired tests with Holm correction, and calibration and selective-prediction analysis.
- **The Core Finding:** A perceptual-hash audit of BRISC 2025 found 1,352 candidate near-duplicate pairs, 439 of them crossing the official train/test split, and 104 training images pixel-identical to test images. On a duplicate-aware split of 5,477 images, frozen DINOv2 ViT-S/14 features were numerically ahead of ImageNet-supervised ResNet-18/50 under an identical linear probe (macro-F1 77.3%, macro-AUROC 93.2%), but no pairwise difference survived Holm correction, and in-pool validation overstated official test macro-F1 by 9.7–13.5 points.
- **The Citation:** Vajjhala, N. R., Ramollari, E., & Kumar, R. (2026). Validation Is Not Enough: A Duplicate-Aware, Reproducible Frozen-Feature Benchmark for Brain Tumour MRI Classification on BRISC 2025. Information, 17(11), Article 1175. https://doi.org/10.3390/info17111175

## Key findings

- **Finding 1.** A perceptual-hash audit of BRISC 2025 found 1,352 candidate near-duplicate pairs, of which 439 crossed the official train/test split; 104 training images were pixel-identical to test images, and 23 candidate pairs carried conflicting labels.
- **Finding 2.** A visual review of sampled pairs showed that most cross-split candidates were other slices of the same scan, with an estimated 5–10% false matches.
- **Finding 3.** Removing redundant copies and every training image connected to the test set (523 images) left a split of 5,477 images that is duplicate-aware but not patient-independent.
- **Finding 4.** Under an identical linear probe, DINOv2 ViT-S/14 was numerically best on every discrimination metric — macro-F1 77.3% against 75.3% for ResNet-18 and 74.6% for ResNet-50, and macro-AUROC 93.2% against 90.4–90.7% — from the smallest embedding of the three (384 dimensions), but no pairwise difference survived Holm correction.
- **Finding 5.** Validation drawn from the training pool overstated official test macro-F1 by 9.7–13.5 points and did not reliably reproduce the test ranking of backbones.
- **Finding 6.** Control experiments showed that the validation–test gap is not an artefact of the filtering: it persisted at every hash threshold and after the removed images were restored, disappeared on pseudo-test sets drawn from the training pool, and mainly reflects a distribution difference between the official partitions.
- **Finding 7.** Meningioma recall stayed at 50–53%, and axial slices were the hardest imaging plane.
- **Finding 8.** All linear heads were under-confident, yet confidence was informative about correctness: accepting only the 50% most confident cases raised accuracy to 91.1% (ResNet-18), 89.7% (ResNet-50) and 93.5% (DINOv2).

Source: Vajjhala et al. (2026), Information, 17(11), Article 1175. https://doi.org/10.3390/info17111175

## The study in detail

**Question.** How much near-duplicate contamination does the BRISC 2025 brain tumour MRI dataset carry across its official train/test split, and what does an honest, low-variance baseline on it look like once that contamination is reduced?

**Finding.** A perceptual-hash audit of the 6,000 BRISC 2025 slices found 1,352 candidate near-duplicate pairs, 439 of them crossing the official train/test split, and 104 training images pixel-identical to test images. On a filtered, duplicate-aware split of 5,477 images, frozen ImageNet-supervised ResNet-18/50 and self-supervised DINOv2 ViT-S/14 features were competitive under an identical linear probe: DINOv2 was numerically ahead (macro-F1 77.3%, macro-AUROC 93.2%), but no pairwise difference survived Holm correction. In-pool validation overstated official test macro-F1 by 9.7–13.5 points.

**Why it matters.** Public brain tumour MRI collections are routinely reported with near-perfect classification accuracy, yet they carry no patient identifiers and their train/test overlap is rarely quantified. The authors argue that on public slice collections without patient identifiers the partition deserves as much scrutiny as the model: duplicates should be audited and documented before splitting, uncertainty and calibration reported rather than a single score, and validation performance interpreted only for the population from which it was drawn.

## Methods and data

| Item | Detail |
|---|---|
| Research questions | Four: how much candidate near-duplication exists within and across the official splits; whether self-supervised DINOv2 features transfer better than supervised ImageNet features once uncertainty is accounted for; whether validation on held-out training-pool data is a reliable proxy for the official test set; and which classes, planes and confidence regimes account for the errors |
| Dataset | BRISC 2025: 6,000 contrast-enhanced T1-weighted MRI slices in four classes (glioma, meningioma, pituitary tumour, no tumour), with an official 5,000/1,000 train/test split and no patient identifiers |
| Audit | A 64-bit perceptual-hash audit with candidate grouping, a threshold-sensitivity analysis, and a visual review of 100 sampled cross-split pairs to estimate precision |
| Split | Redundant copies and every training image connected to a test image removed (523 images), leaving 5,477 images; duplicate-aware but not patient-independent |
| Models | Frozen ResNet-18 and ResNet-50 (ImageNet-supervised) and DINOv2 ViT-S/14 (self-supervised), each with an identical linear probe over three seeds |
| Inference | Group-level bootstrap confidence intervals, paired tests and Holm multiplicity correction; expected calibration error and selective-prediction (risk–coverage) analysis |
| Main result | DINOv2 numerically ahead (macro-F1 77.3 ± 0.1%, macro-AUROC 93.2%) but not significantly so; in-pool validation overstated official test macro-F1 by 9.7–13.5 points; meningioma recall stayed at 50–53% |
| Released | The splits, the audit tables and the per-image predictions |

## Definitions and key terms

- **BRISC 2025:** A public brain tumour MRI dataset of 6,000 contrast-enhanced T1-weighted slices in four classes — glioma, meningioma, pituitary tumour and no tumour — assembled from pre-existing public collections and released with an official 5,000/1,000 train/test split and no patient identifiers. (as used in Vajjhala et al. (2026))
- **Near-duplicate contamination:** The presence of the same or nearly the same image — a re-exported copy, a lightly cropped duplicate or an adjacent slice of the same scan — on both sides of a train/test split, which makes test scores optimistic. (as used in Vajjhala et al. (2026))
- **Linear probe:** An evaluation protocol in which a pre-trained encoder is kept frozen and only a linear classifier is trained on its features, which isolates the quality of the representation from fine-tuning choices. (as used in Vajjhala et al. (2026))
- **Duplicate-aware split:** In this article, a split from which redundant copies and every training image connected by the perceptual-hash audit to a test image have been removed. It reduces leakage but, without patient identifiers, cannot guarantee that the two sides hold different patients. (as used in Vajjhala et al. (2026))
- **Selective prediction:** Accepting a model’s prediction only when its confidence is high enough and deferring the remaining cases; the article reports accuracy at given coverage levels and the area under the risk–coverage curve (AURC). (as used in Vajjhala et al. (2026))

## Abstract

Public brain tumour MRI collections are routinely reported with near-perfect classification accuracy, yet they carry no patient identifiers and their train/test overlap is rarely quantified. We present a duplicate-aware, reproducible frozen-feature benchmark on BRISC 2025 (6000 contrast-enhanced T1-weighted slices; glioma, meningioma, pituitary tumour, no tumour). A perceptual-hash audit found 1352 candidate near-duplicate pairs, with 439 of them crossing the official train/test split, and 104 training images that are pixel-identical to test images; a visual review of sampled pairs showed that most candidates were other slices of the same scan, with an estimated 5–10% false matches. Removing redundant copies and every training image connected to the test set left 5477 images, a split that is duplicate-aware but not patient-independent. Under an identical linear probe, ImageNet-supervised ResNet-18/50 and self-supervised DINOv2 ViT-S/14 were competitive: DINOv2 was numerically ahead (macro-F1 77.3 ± 0.1%, macro-AUROC 93.2%), but no pairwise difference survived Holm correction. In-pool validation overstated official test macro-F1 by 9.7–13.5 points; control experiments showed that this gap is not an artefact of the filtering but mainly reflects a distribution difference between the official partitions. Meningioma recall stayed at 50–53%, and all linear heads were under-confident. We release the splits, audit tables and per-image predictions.

## Limitations

- The evaluation is duplicate-aware but not patient-independent, and the filtering may discard genuinely distinct but visually similar images.
- The 64-bit perceptual hash errs in both directions — it under-detects rotated, reflected or heavily cropped duplicates, and an estimated 5–10% of cross-split candidate pairs connect different patients — so the contamination figures are approximate; the precision estimate rests on a visual review of 100 sampled pairs.
- Results on the filtered test set are not numerically comparable with results on the original 1,000-image test set.
- One linear protocol was evaluated at one resolution, without a hyper-parameter search, augmentation or fine-tuning, so the absolute numbers are a floor rather than a ceiling.
- The comparison covers one self-supervised generation only; a planned DINOv3 run could not be completed.
- No clinical validity is demonstrated; the authors state that the models reported must not be used for clinical decisions.

## Practical implications

- Researchers benchmarking on BRISC 2025 can use the released splits, audit tables and per-image predictions as a common, documented basis for comparison.
- For researchers reporting results on public slice collections, the authors recommend reporting uncertainty and calibration rather than a single score.
- For researchers selecting models on validation data, in-pool validation estimates performance on data like the training pool, not on the official test set: here it overstated official test macro-F1 by 9.7–13.5 points.
- For clinicians and developers, no clinical validity is demonstrated, and the authors state that the models reported must not be used for clinical decisions.

## When this research may be relevant

This article may be relevant to researchers working on brain tumour MRI classification, on data leakage, near-duplicate images and train/test contamination in public medical imaging datasets, and on benchmarking frozen foundation-model features such as DINOv2 against ImageNet-supervised networks. It also speaks to work on calibration, selective prediction and the reliability of validation scores as a proxy for test performance, and it is a reusable audit protocol for public image collections released without patient identifiers.

## How to cite

**APA 7:** Vajjhala, N. R., Ramollari, E., & Kumar, R. (2026). Validation Is Not Enough: A Duplicate-Aware, Reproducible Frozen-Feature Benchmark for Brain Tumour MRI Classification on BRISC 2025. Information, 17(11), Article 1175. https://doi.org/10.3390/info17111175

**MLA 9:** Vajjhala, Narasimha Rao, et al. “Validation Is Not Enough: A Duplicate-Aware, Reproducible Frozen-Feature Benchmark for Brain Tumour MRI Classification on BRISC 2025.” Information, vol. 17, no. 11, 2026, article 1175. https://doi.org/10.3390/info17111175.

**IEEE:** N. R. Vajjhala, E. Ramollari, and R. Kumar, “Validation Is Not Enough: A Duplicate-Aware, Reproducible Frozen-Feature Benchmark for Brain Tumour MRI Classification on BRISC 2025,” Information, vol. 17, no. 11, Art. no. 1175, Nov. 2026, doi: 10.3390/info17111175.

```bibtex
@article{vajjhala2026duplicate,
  title = {Validation Is Not Enough: A Duplicate-Aware, Reproducible Frozen-Feature Benchmark for Brain Tumour MRI Classification on BRISC 2025},
  author = {Vajjhala, Narasimha Rao and Ramollari, Ervin and Kumar, Rajesh},
  journal = {Information},
  volume = {17},
  number = {11},
  pages = {1175},
  year = {2026},
  publisher = {MDPI},
  doi = {10.3390/info17111175},
  url = {https://doi.org/10.3390/info17111175}
}
```

## Related publications

- Ramollari et al. (2026). [Breast Ultrasound Classification Under Cross-Dataset Shift: A Multi-Seed Benchmark of Convolutional Networks, a Vision Transformer, and a Self-Supervised Foundation Model](https://www.narasimharao.net/research/breast-ultrasound-classification-cross-dataset-shift-multi-seed-benchmark/). https://doi.org/10.3390/a19111011. Trained on BUS-BRA and evaluated unchanged on BUSI, the supervised models largely retained their discrimination, but in every official-fold run ResNet-50 and ViT-B/16 labelled a larger share of BUSI than of BUS-BRA images malignant although prevalence was the same, a shift that temperature scaling cannot correct.
- Haveri & Vajjhala (2026). [Deep Learning for Medical Image Analysis: CNN-based Pneumonia Detection on Chest X-Rays](https://www.narasimharao.net/research/cnn-pneumonia-detection-chest-xray-efficientnetv2/). https://doi.org/10.1109/ICPCSN68523.2026.11543623. A lightweight EfficientNetV2B0 model trained under CLAIM and TRIPOD-AI reporting standards achieved an AUC of 0.967 (95% CI 0.953–0.979) and pneumonia recall of 0.98 on paediatric chest X-rays, missing only about 2% of true cases.
- Kumar et al. (2026). [A Comparative Evaluation of Deep Learning Architectures for Weed Classification, with an Exploratory Out-of-Distribution Analysis of Albanian Field Images](https://www.narasimharao.net/research/deep-learning-weed-classification-deepweeds-albania/). https://doi.org/10.3390/info17080751. Fine-tuned DINOv2 and EfficientNet-B0 performed best on nine-class DeepWeeds data, but differences among several high-performing models were not consistently statistically significant across random seeds.
- Vajjhala & Eappen (2024). [Data Envelopment Analysis in Healthcare Management: Overview of the Latest Trends](https://www.narasimharao.net/research/data-envelopment-analysis-healthcare-management-review/). https://doi.org/10.4018/979-8-3693-0255-2.ch011. The review shows that DEA is widely applied in hospital management, nursing and outpatient services for efficiency measurement, benchmarking, resource allocation and performance evaluation, and identifies six limitations, including input/output selection, outlier sensitivity, inability to handle statistical noise and the static nature of traditional models.
- Eappen et al. (2026). [Enhancing Healthcare Informatics with Transparent and Explainable AI](https://www.narasimharao.net/research/enhancing-healthcare-informatics-transparent-explainable-ai/). https://doi.org/10.1201/9781003604440. The volume applies explainable and transparent AI across clinical decision-making, telehealth, electronic health records, EEG analysis, wearable monitoring and mental health, pairing ethical analysis with applied systems and strategies for building trust.
