Open-access journal article · 2026

Validation Is Not Enough: A Duplicate-Aware, Reproducible Frozen-Feature Benchmark for Brain Tumour MRI Classification on BRISC 2025

Narasimha Rao VajjhalaiD, Ervin RamollariiD & Rajesh KumariD

Information, 17(11), Article 1175 · Published · Special Issue Applications of Deep Learning in Bioinformatics and Image Processing, 2nd Edition

Scopus Q1 (CiteScore 8.2)Web of Science ESCI Q2Impact Factor 4.3 (JCR 2025)Open access · CC BY 4.0

Summary

Vajjhala et al. (2026) audited the BRISC 2025 brain tumour MRI dataset for near-duplicate images with a perceptual hash and benchmarked frozen ResNet-18, ResNet-50 and DINOv2 ViT-S/14 features under an identical linear probe on a duplicate-aware split of 5,477 images. The audit found 1,352 candidate near-duplicate pairs, 439 of them crossing the official train/test split, and 104 training images pixel-identical to test images. DINOv2 was numerically ahead (macro-F1 77.3%, macro-AUROC 93.2%), but no pairwise difference survived Holm correction, and in-pool validation overstated official test macro-F1 by 9.7–13.5 points.

Practical implication. Researchers using public MRI slice collections without patient identifiers should audit and document duplicates before splitting, the authors argue, because the partition deserves as much scrutiny as the model.

The Problem
Public brain tumour MRI collections are routinely reported with near-perfect classification accuracy, yet they carry no patient identifiers and their train/test overlap is rarely quantified.
The Methodology
Dataset audit and controlled benchmark: a perceptual-hash near-duplicate audit of the BRISC 2025 brain tumour MRI dataset (6,000 slices), a duplicate-aware group-aware split of 5,477 images, and a linear probe on frozen ResNet-18, ResNet-50 and DINOv2 ViT-S/14 features over three seeds, with group-level bootstrap confidence intervals, paired tests with Holm correction, and calibration and selective-prediction analysis.
The Core Finding
A perceptual-hash audit of BRISC 2025 found 1,352 candidate near-duplicate pairs, 439 of them crossing the official train/test split, and 104 training images pixel-identical to test images. On a duplicate-aware split of 5,477 images, frozen DINOv2 ViT-S/14 features were numerically ahead of ImageNet-supervised ResNet-18/50 under an identical linear probe (macro-F1 77.3%, macro-AUROC 93.2%), but no pairwise difference survived Holm correction, and in-pool validation overstated official test macro-F1 by 9.7–13.5 points.
The Citation
Vajjhala, N. R., Ramollari, E., & Kumar, R. (2026). Validation Is Not Enough: A Duplicate-Aware, Reproducible Frozen-Feature Benchmark for Brain Tumour MRI Classification on BRISC 2025. Information, 17(11), Article 1175. https://doi.org/10.3390/info17111175

The summary, key findings, methods, implications and relevance notes on this page are this website’s own description of the paper, written from the published abstract and text. The official abstract and citation details are given further down.

Key findings

  1. Finding 1. A perceptual-hash audit of BRISC 2025 found 1,352 candidate near-duplicate pairs, of which 439 crossed the official train/test split; 104 training images were pixel-identical to test images, and 23 candidate pairs carried conflicting labels.
  2. Finding 2. A visual review of sampled pairs showed that most cross-split candidates were other slices of the same scan, with an estimated 5–10% false matches.
  3. Finding 3. Removing redundant copies and every training image connected to the test set (523 images) left a split of 5,477 images that is duplicate-aware but not patient-independent.
  4. Finding 4. Under an identical linear probe, DINOv2 ViT-S/14 was numerically best on every discrimination metric — macro-F1 77.3% against 75.3% for ResNet-18 and 74.6% for ResNet-50, and macro-AUROC 93.2% against 90.4–90.7% — from the smallest embedding of the three (384 dimensions), but no pairwise difference survived Holm correction.
  5. Finding 5. Validation drawn from the training pool overstated official test macro-F1 by 9.7–13.5 points and did not reliably reproduce the test ranking of backbones.
  6. Finding 6. Control experiments showed that the validation–test gap is not an artefact of the filtering: it persisted at every hash threshold and after the removed images were restored, disappeared on pseudo-test sets drawn from the training pool, and mainly reflects a distribution difference between the official partitions.
  7. Finding 7. Meningioma recall stayed at 50–53%, and axial slices were the hardest imaging plane.
  8. Finding 8. All linear heads were under-confident, yet confidence was informative about correctness: accepting only the 50% most confident cases raised accuracy to 91.1% (ResNet-18), 89.7% (ResNet-50) and 93.5% (DINOv2).

Source: Vajjhala et al. (2026), Information, 17(11), Article 1175. DOI: 10.3390/info17111175

The study in detail

What question does this paper answer?

How much near-duplicate contamination does the BRISC 2025 brain tumour MRI dataset carry across its official train/test split, and what does an honest, low-variance baseline on it look like once that contamination is reduced?

What did the study find, in detail?

A perceptual-hash audit of the 6,000 BRISC 2025 slices found 1,352 candidate near-duplicate pairs, 439 of them crossing the official train/test split, and 104 training images pixel-identical to test images. On a filtered, duplicate-aware split of 5,477 images, frozen ImageNet-supervised ResNet-18/50 and self-supervised DINOv2 ViT-S/14 features were competitive under an identical linear probe: DINOv2 was numerically ahead (macro-F1 77.3%, macro-AUROC 93.2%), but no pairwise difference survived Holm correction. In-pool validation overstated official test macro-F1 by 9.7–13.5 points.

Why does it matter?

Public brain tumour MRI collections are routinely reported with near-perfect classification accuracy, yet they carry no patient identifiers and their train/test overlap is rarely quantified. The authors argue that on public slice collections without patient identifiers the partition deserves as much scrutiny as the model: duplicates should be audited and documented before splitting, uncertainty and calibration reported rather than a single score, and validation performance interpreted only for the population from which it was drawn.

Methods and data

Design, data and results of Validation Is Not Enough: A Duplicate-Aware, Reproducible Frozen-Feature Benchmark for Brain Tumour MRI Classification on BRISC 2025
Research questionsFour: how much candidate near-duplication exists within and across the official splits; whether self-supervised DINOv2 features transfer better than supervised ImageNet features once uncertainty is accounted for; whether validation on held-out training-pool data is a reliable proxy for the official test set; and which classes, planes and confidence regimes account for the errors
DatasetBRISC 2025: 6,000 contrast-enhanced T1-weighted MRI slices in four classes (glioma, meningioma, pituitary tumour, no tumour), with an official 5,000/1,000 train/test split and no patient identifiers
AuditA 64-bit perceptual-hash audit with candidate grouping, a threshold-sensitivity analysis, and a visual review of 100 sampled cross-split pairs to estimate precision
SplitRedundant copies and every training image connected to a test image removed (523 images), leaving 5,477 images; duplicate-aware but not patient-independent
ModelsFrozen ResNet-18 and ResNet-50 (ImageNet-supervised) and DINOv2 ViT-S/14 (self-supervised), each with an identical linear probe over three seeds
InferenceGroup-level bootstrap confidence intervals, paired tests and Holm multiplicity correction; expected calibration error and selective-prediction (risk–coverage) analysis
Main resultDINOv2 numerically ahead (macro-F1 77.3 ± 0.1%, macro-AUROC 93.2%) but not significantly so; in-pool validation overstated official test macro-F1 by 9.7–13.5 points; meningioma recall stayed at 50–53%
ReleasedThe splits, the audit tables and the per-image predictions
CitationVajjhala et al. (2026) · DOI 10.3390/info17111175

Definitions and key terms

BRISC 2025
A public brain tumour MRI dataset of 6,000 contrast-enhanced T1-weighted slices in four classes — glioma, meningioma, pituitary tumour and no tumour — assembled from pre-existing public collections and released with an official 5,000/1,000 train/test split and no patient identifiers. — as used in Vajjhala et al. (2026)
Near-duplicate contamination
The presence of the same or nearly the same image — a re-exported copy, a lightly cropped duplicate or an adjacent slice of the same scan — on both sides of a train/test split, which makes test scores optimistic. — as used in Vajjhala et al. (2026)
Linear probe
An evaluation protocol in which a pre-trained encoder is kept frozen and only a linear classifier is trained on its features, which isolates the quality of the representation from fine-tuning choices. — as used in Vajjhala et al. (2026)
Duplicate-aware split
In this article, a split from which redundant copies and every training image connected by the perceptual-hash audit to a test image have been removed. It reduces leakage but, without patient identifiers, cannot guarantee that the two sides hold different patients. — as used in Vajjhala et al. (2026)
Selective prediction
Accepting a model’s prediction only when its confidence is high enough and deferring the remaining cases; the article reports accuracy at given coverage levels and the area under the risk–coverage curve (AURC). — as used in Vajjhala et al. (2026)

All key terms across the publications →

Abstract

Public brain tumour MRI collections are routinely reported with near-perfect classification accuracy, yet they carry no patient identifiers and their train/test overlap is rarely quantified. We present a duplicate-aware, reproducible frozen-feature benchmark on BRISC 2025 (6000 contrast-enhanced T1-weighted slices; glioma, meningioma, pituitary tumour, no tumour). A perceptual-hash audit found 1352 candidate near-duplicate pairs, with 439 of them crossing the official train/test split, and 104 training images that are pixel-identical to test images; a visual review of sampled pairs showed that most candidates were other slices of the same scan, with an estimated 5–10% false matches. Removing redundant copies and every training image connected to the test set left 5477 images, a split that is duplicate-aware but not patient-independent. Under an identical linear probe, ImageNet-supervised ResNet-18/50 and self-supervised DINOv2 ViT-S/14 were competitive: DINOv2 was numerically ahead (macro-F1 77.3 ± 0.1%, macro-AUROC 93.2%), but no pairwise difference survived Holm correction. In-pool validation overstated official test macro-F1 by 9.7–13.5 points; control experiments showed that this gap is not an artefact of the filtering but mainly reflects a distribution difference between the official partitions. Meningioma recall stayed at 50–53%, and all linear heads were under-confident. We release the splits, audit tables and per-image predictions.

Abstract as published in Information under a Creative Commons licence.

Keywords: brain tumour MRI classification; benchmarking; reproducibility; data leakage; near-duplicate detection; foundation models; DINOv2; linear probing; calibration; selective prediction

Limitations

  • The evaluation is duplicate-aware but not patient-independent, and the filtering may discard genuinely distinct but visually similar images.
  • The 64-bit perceptual hash errs in both directions — it under-detects rotated, reflected or heavily cropped duplicates, and an estimated 5–10% of cross-split candidate pairs connect different patients — so the contamination figures are approximate; the precision estimate rests on a visual review of 100 sampled pairs.
  • Results on the filtered test set are not numerically comparable with results on the original 1,000-image test set.
  • One linear protocol was evaluated at one resolution, without a hyper-parameter search, augmentation or fine-tuning, so the absolute numbers are a floor rather than a ceiling.
  • The comparison covers one self-supervised generation only; a planned DINOv3 run could not be completed.
  • No clinical validity is demonstrated; the authors state that the models reported must not be used for clinical decisions.

Practical implications

  • Researchers benchmarking on BRISC 2025 can use the released splits, audit tables and per-image predictions as a common, documented basis for comparison.
  • For researchers reporting results on public slice collections, the authors recommend reporting uncertainty and calibration rather than a single score.
  • For researchers selecting models on validation data, in-pool validation estimates performance on data like the training pool, not on the official test set: here it overstated official test macro-F1 by 9.7–13.5 points.
  • For clinicians and developers, no clinical validity is demonstrated, and the authors state that the models reported must not be used for clinical decisions.

Drawn from Vajjhala et al. (2026), Information, 17(11), Article 1175.

When this research may be relevant

This article may be relevant to researchers working on brain tumour MRI classification, on data leakage, near-duplicate images and train/test contamination in public medical imaging datasets, and on benchmarking frozen foundation-model features such as DINOv2 against ImageNet-supervised networks. It also speaks to work on calibration, selective prediction and the reliability of validation scores as a proxy for test performance, and it is a reusable audit protocol for public image collections released without patient identifiers.

Research topics addressed: brain tumour MRI classification; medical image analysis; deep learning; benchmarking; data leakage; near-duplicate detection; perceptual hashing; train/test contamination; foundation models; DINOv2; ResNet; linear probing; frozen features; calibration; selective prediction; reproducibility; BRISC 2025 dataset

How to cite

APA 7
Vajjhala, N. R., Ramollari, E., & Kumar, R. (2026). Validation Is Not Enough: A Duplicate-Aware, Reproducible Frozen-Feature Benchmark for Brain Tumour MRI Classification on BRISC 2025. Information, 17(11), Article 1175. https://doi.org/10.3390/info17111175
MLA 9
Vajjhala, Narasimha Rao, et al. “Validation Is Not Enough: A Duplicate-Aware, Reproducible Frozen-Feature Benchmark for Brain Tumour MRI Classification on BRISC 2025.” Information, vol. 17, no. 11, 2026, article 1175. https://doi.org/10.3390/info17111175.
IEEE
N. R. Vajjhala, E. Ramollari, and R. Kumar, “Validation Is Not Enough: A Duplicate-Aware, Reproducible Frozen-Feature Benchmark for Brain Tumour MRI Classification on BRISC 2025,” Information, vol. 17, no. 11, Art. no. 1175, Nov. 2026, doi: 10.3390/info17111175.
BibTeX
@article{vajjhala2026duplicate,
  title = {Validation Is Not Enough: A Duplicate-Aware, Reproducible Frozen-Feature Benchmark for Brain Tumour MRI Classification on BRISC 2025},
  author = {Vajjhala, Narasimha Rao and Ramollari, Ervin and Kumar, Rajesh},
  journal = {Information},
  volume = {17},
  number = {11},
  pages = {1175},
  year = {2026},
  publisher = {MDPI},
  doi = {10.3390/info17111175},
  url = {https://doi.org/10.3390/info17111175}
}
RIS
TY  - JOUR
TI  - Validation Is Not Enough: A Duplicate-Aware, Reproducible Frozen-Feature Benchmark for Brain Tumour MRI Classification on BRISC 2025
AU  - Vajjhala, Narasimha Rao
AU  - Ramollari, Ervin
AU  - Kumar, Rajesh
JO  - Information
PY  - 2026
DA  - 2026/11/19
VL  - 17
IS  - 11
SP  - 1175
PB  - MDPI
SN  - 2078-2489
DO  - 10.3390/info17111175
UR  - https://www.narasimharao.net/research/duplicate-aware-frozen-feature-benchmark-brain-tumour-mri-brisc-2025/
LA  - en
ER  - 
Download citation:BibTeXRISCSL-JSONMarkdown

Related research

2026
Ramollari et al. (2026) · Algorithms · DOI 10.3390/a19111011

Trained on BUS-BRA and evaluated unchanged on BUSI, the supervised models largely retained their discrimination, but in every official-fold run ResNet-50 and ViT-B/16 labelled a larger share of BUSI than of BUS-BRA images malignant although prevalence was the same, a shift that temperature scaling cannot correct.

2026
Haveri & Vajjhala (2026) · 2026 6th International Conference on Pervasive Computing and Social Networking (ICPCSN) · DOI 10.1109/ICPCSN68523.2026.11543623

A lightweight EfficientNetV2B0 model trained under CLAIM and TRIPOD-AI reporting standards achieved an AUC of 0.967 (95% CI 0.953–0.979) and pneumonia recall of 0.98 on paediatric chest X-rays, missing only about 2% of true cases.

2026
Kumar et al. (2026) · Information · DOI 10.3390/info17080751

Fine-tuned DINOv2 and EfficientNet-B0 performed best on nine-class DeepWeeds data, but differences among several high-performing models were not consistently statistically significant across random seeds.

2024
Vajjhala & Eappen (2024) · Data Envelopment Analysis (DEA) Methods for Maximizing Efficiency · DOI 10.4018/979-8-3693-0255-2.ch011

The review shows that DEA is widely applied in hospital management, nursing and outpatient services for efficiency measurement, benchmarking, resource allocation and performance evaluation, and identifies six limitations, including input/output selection, outlier sensitivity, inability to handle statistical noise and the static nature of traditional models.

2026
Eappen et al. (2026) · Auerbach Publications (CRC Press) · DOI 10.1201/9781003604440

The volume applies explainable and transparent AI across clinical decision-making, telehealth, electronic health records, EEG analysis, wearable monitoring and mental health, pairing ethical analysis with applied systems and strategies for building trust.

All publication summaries → · All selected publications →