Reading the PathoROB robustness index with fixed measurements: a no-signal level, a colour reference, and what 48 named numbers explain in a foundation model's hospital signal

Parallax Pathology · Technical note

Russell Parrish · analysis carried out with AI assistants · Version 1, 10 November 2026

1. Question

PathoROB asks whether a foundation model's representation is organised by biology or by medical centre. Its Robustness Index (RI) compares, among each tile's nearest neighbours, those with the same tissue class from another centre against those with another tissue class from the same centre [1]. The published leaderboard lists 26 models.

The study began with one question, written down before any tile was measured: where does a fixed, training-free instrument sit on this index? The answer led to three further questions, each taken in turn:

• What does the index score for features with no signal, and for plain colour?
• Can fixed, named measurements explain, and remove, a foundation model's hospital signal?
• Does that removal hold on hospitals the fit never saw, and how does it compare with a standard batch correction?

2. Setup

The instrument returned a valid status on every tile. Eight of its values were undefined (seven in Tolkach, one in Camelyon) and were set to the dataset mean, as planned.

3. Reference rows

3.1 The index for features with no signal

The no-signal level is not the same number on each dataset. The index compares counts of two kinds of neighbour, and how often each kind occurs for unrelated tiles depends on the dataset's mix of classes and centres. A score of 0.60 therefore sits 0.10 above no signal on TCGA and 0.30 above it on Tolkach, and an average over the three datasets mixes those scales.

On Camelyon, the 11 models below the random-feature level are MUSK, Prov-GigaPath, RETCCL, RudolfV, Kaiko ViT-B/8, UNI, Ciga, CTransPath, Kang-DINO, Phikon-v2 and Phikon (0.467 down to 0.011). For these, neighbours follow the hospital more than they would for unrelated tiles.

3.2 Colour as a reference

Six colour numbers score below every published model on Camelyon (0.009 against a lowest published value of 0.011), and below the random-feature level on all three datasets. They are not uninformative: their tissue balanced accuracy is well above the random-feature values in section 3.4. On Camelyon, colour separates tissue and separates hospitals, and the hospitals dominate the neighbourhoods.

3.3 A view relative to no signal

Two quantities read against the random-feature run on the same dataset:

• Tissue signal beyond chance = (balanced accuracy − random-feature value) / (1 − random-feature value).
• Hospital pull beyond chance = random-feature RI − RI. Positive means neighbours follow the hospital more than they would for unrelated tiles.

On Camelyon, Phikon-v2's hospital pull (0.48) is within 0.01 of that of six colour numbers (0.49), while its tissue signal is far higher (0.91 against 0.60). The same model sits well on the robust side on TCGA and Tolkach. This view shows the two properties the leaderboard value folds together.

3.4 The fixed instrument

The plan predicted that the instrument would sit low on average, plausibly below most foundation models, and that Camelyon was the dataset where it could land among them. Both held: rank 23 of 27 on average, above 11 of 26 models on Camelyon. The instrument is built to be sensitive to how a slide was handled, so a low index is its expected operating window on this benchmark. Its tissue signal is modest beside the foundation models'.

3.5 Predicting the hospital from fixed numbers

Logistic regression predicting the medical centre, five folds grouped by slide (by case for TCGA), all tiles. Accuracy / balanced accuracy / one-vs-rest AUC:

Colour alone matches or exceeds the instrument alone on accuracy and AUC on every dataset. Adding the 42 measurements to colour raises balanced accuracy on all three (0.15 to 0.27, 0.64 to 0.83, 0.81 to 0.87). The benchmark paper reports that the centre can be predicted from foundation-model features with 88% to 98% mean accuracy [1]; its protocol may differ from this one.

4. What fixed measurements explain in a foundation model

4.1 Design

The plan was written and hashed before any Phikon-v2 feature was extracted. Per dataset, every one of Phikon-v2's feature dimensions is regressed on a set Z of fixed measurements (ordinary least squares with intercept, Z standardised, all tiles of that dataset, no labels of any kind). The residual is kept and scored by the benchmark, unchanged. The sets:

• Instrument (42 columns), colour (6), both (48).
• Two small named subsets: five axes that had failed to carry between laboratories in separate work (section 6), and the axes that were most hospital-driven in the other two datasets (3 to 20 axes).
• Null: 42 random Gaussian columns, three seeds, to show what removing 42 directions of anything does.

Criterion, fixed in advance. A removal works on a dataset if its RI gain exceeds the best random-column gain by at least 0.05 and tissue balanced accuracy stays within 0.02 of the unmodified model. Prediction. Removing colour raises the Camelyon index. No direction was claimed for the instrument beyond colour.

4.2 Explained variance

Forty-eight named numbers predict 12.7%, 30.9% and 27.5% of the model's feature variance on TCGA, Camelyon and Tolkach. Random columns predict 0.1% to 0.3%. The instrument alone predicts more than colour alone on every dataset (25.1% against 20.5% on Camelyon).

4.3 Removal

Rank places each average against the 26 published rows, which include Phikon-v2 itself. Centre AUC is the benchmark's own linear hospital-prediction score; it reflects folds below 0.5, so noise scores above 0.5.

Best random-column gain: −0.005, +0.006 and +0.003.

• The prediction held. Removing six colour numbers moved Phikon-v2's Camelyon index from 0.019 to 0.411, with tissue accuracy unchanged (0.954 to 0.952).
• Colour carries most of the gain. It met the criterion on all three datasets by itself.
• The instrument beyond colour (gain of both minus gain of colour): +0.083 on Camelyon, +0.004 on TCGA, +0.003 on Tolkach. Clear on Camelyon only.
• Few named axes are not enough. The small subsets carried little of the effect; the full 42-column block did.
• Two readouts disagree in an informative way. On Camelyon, removing the instrument lowered the model's linear hospital-prediction AUC more than removing colour did (1.000 to 0.646, against 0.865) while raising the index less (0.361 against 0.411). Linear separability of hospitals and the purity of neighbourhoods respond differently to the same removal.

5. Does the removal hold on hospitals it did not see?

Section 4 fitted the removal on the same tiles it was scored on, as ComBat is fitted. Two checks were planned and hashed before they ran.

5.1 A design that failed its own null

The first held-out design fitted a separate removal for each hospital from the other hospitals. Removing 42 random columns this way lowered Phikon-v2's index (Tolkach 0.768 to 0.589, Camelyon 0.019 to 0.000). The design itself injects hospital structure: each hospital is shifted by a fit that excludes it, so hospitals are shifted by different amounts. Those numbers are kept and not interpreted. The redesign below was written and hashed after this fault was found and before it ran, and is declared as post hoc.

5.2 One fit, applied to every tile

• Other datasets. Fit on the pooled tiles of the other two datasets; apply to the scored dataset. No tile or hospital of the scored dataset is seen.
• Half the hospitals. Fit on every second centre of a dataset (sorted by name: 13 of 26, 3 of 5, 2 of 4); apply to all its tiles.

• Fitted on other datasets, nothing transfers. Every gain is within 0.022 of zero, and none meets the criterion.
• Fitted on half the hospitals, a gain survives on Camelyon only. The instrument reached 0.269 and colour 0.163, both meeting the criterion (random columns: 0.066). On TCGA no variant met it; on Tolkach the index fell.
• The instrument held up better than colour on Camelyon. It kept 73% of its in-sample gain against 37% for colour. The plan had predicted that colour would keep most of its gain; that did not hold.
• Limits of the half-hospital fit. On Tolkach it uses 2 of 4 centres, one of them derived from TCGA.

5.3 Matched ComBat

ComBat [7] as in the benchmark paper: the pycombat implementation (inmoose 0.8.1), batch set to the medical centre, no biological covariates, fitted on all tiles of each dataset. Our values lie within 0.03 of one column of the paper's table for Phikon-v2 on all three datasets (0.784, 0.446, 0.923), which we take to be the paper's ComBat column; the extracted table header was ambiguous.

On Camelyon the label-free removal is above ComBat (0.494 against 0.424); on TCGA and Tolkach ComBat is higher. Applying the removal before ComBat adds 0.098 on Camelyon and 0.009 on TCGA, and costs 0.039 on Tolkach. Both methods here are fitted on the scored tiles. The benchmark paper reports that ComBat did not consistently help downstream models [1]; downstream training is outside this note.

6. Which named axes follow the hospital

For each of the 42 axes, the share of between-group variance that follows the medical centre and not the tissue class was computed per dataset and averaged. This is a one-way variance share per axis, not the neighbour index, and centre and class are not fully balanced in every dataset.

• Most hospital-driven: delta_r, gap_ds, blob_size_mean, g3_infl, gamma, k_rho, k_ds, k_rv.
• Least hospital-driven: g4_slope, g4_s32, g4_s8, omega, g1_curv, rc_s, rc_f, b_mu.
• Agreement with separate work. A per-field map between the Kather colorectal collection and SurGen slides [8], on different data and a different contrast, had marked five of these axes as breaking and six as holding. The five breaking axes rank 5, 6, 8, 10 and 11 here. The probability that five randomly chosen axes all fall in the top 11 of 42 is 0.0005. Of the six holding axes, five rank 27 or lower; one ranks 13.
• Selecting axes by this ranking helped a little in one direction and not in the other. Dropping the five breaking axes raised the instrument's average index from 0.510 to 0.547 (0.518, 0.570, 0.551), almost all of it on Camelyon (0.471 to 0.570). Keeping only the six holding axes lowered it to 0.475 (0.534, 0.479, 0.413), with Tolkach tissue accuracy falling to 0.366. The plan had predicted that the six would be at least as robust as the full instrument; that did not hold.

7. Reading

• A random-feature row gives the index a zero. Without it, 0.47 on Camelyon and 0.30 on Tolkach both look poor; with it, both are at or near the no-signal level.
• The index does not include tissue signal. Random features reach about 0.5 on two datasets with no tissue information. A representation can score near the no-signal level by carrying nothing, or by carrying tissue and hospital in balance.
• Much of one model's hospital signal is nameable. On Camelyon, 48 fixed numbers predict 31% of Phikon-v2's feature variance, and the model's hospital pull matches that of six colour numbers. Removing the predicted part moves the model's index from 0.019, beside the colour reference, to 0.494, the no-signal level, with tissue accuracy within 0.004.
• That is a description, not a correction. The removal works when fitted on the scored tiles, partly when fitted on half of Camelyon's hospitals, and not at all when fitted on other datasets. ComBat, which uses hospital labels, does better on average.
• Camelyon is the instrument's window. The 42 measurements add beyond colour clearly on Camelyon only, under the in-sample fit (+0.083) and the half-hospital fit (+0.086). An earlier pre-registered test on Camelyon17-WILDS found the same block adding to colour for tumour detection across held-out hospitals, and not in a kidney cohort [9]. Both use Camelyon-derived tiles, so they are two tasks on related data, not two independent datasets.
• Site-sensitive axes replicate. The axes that broke between two colorectal laboratories are among the most hospital-driven here, on three other datasets.

8. Limitations

• One foundation model was read (Phikon-v2). The removal is linear and takes at most 48 directions out of a 2,048-dimensional feature.
• The removal in section 4 and ComBat are both fitted on the scored tiles. Held-out results are in section 5 and are much weaker.
• The first held-out design was faulty and was replaced by a redesign declared post hoc. The paper's ComBat column was identified from our own result.
• The random-feature level was added after the instrument's placement was seen. The relative view (section 3.3) and the hospital-prediction analysis (section 3.5) were proposed after the first results and have no hashed plan. The instrument's placement, the removal and the checks each had one.
• Random features were 42-dimensional. The no-signal level was not checked at the dimensions of the published models.
• Per-dataset standardisation of the reference columns is our choice; another scaling would change the colour and instrument rows.
• Foundation-model values in section 3.3 come from the results shipped in the benchmark repository; leaderboard values are as listed at the stated commit.
• "Hospital pull" is a difference of index values, not a causal quantity. Matching colour's pull does not show that colour is the cause.
• The index's no-signal level differs by dataset, so averages and ranks in this note inherit the mixing of scales described in section 3.1.
• We did not find a random-feature reference for the index in the benchmark's preprint text; one may exist in material we did not read.

Data and code availability

Hashed plans, scripts, per-run index summaries, per-axis tables and logs are held by the authors (folder pathorob-core42-20261109; this note in PATHOROB_NOTE_20261110) and are available on request. The benchmark code and datasets are public [1]; the instrument is public [2]; Phikon-v2's weights are public [3]. The TCGA tile subset used by the benchmark is licensed for non-commercial use (CC BY-NC-SA 4.0).

References

1. Kömen J, de Jong ED, Hense J, Marienwald H, Dippel J, Naumann P, Marcus E, Ruff L, Alber M, Teuwen J, Klauschen F, Müller K-R. Towards robust foundation models for digital pathology. Nature Communications 2026;17:5218. doi:10.1038/s41467-026-73923-2. Preprint: arXiv:2507.17845. Code and leaderboard: https://github.com/bifold-pathomics/PathoROB (commit d81a04b).
2. Parrish R. parallax-core42 v1.0.0: a fixed 42-axis H&E measurement instrument. https://github.com/parallaxmetrology/parallax-core42
3. Filiot A, Jacob P, Mac Kain A, Saillard C. Phikon-v2, a large and public feature extractor for biomarker prediction. arXiv:2409.09173, 2024.
4. Bándi P, et al. From detection of individual metastases to classification of lymph node status at the patient level: the CAMELYON17 challenge. IEEE Transactions on Medical Imaging 2019;38(2):550–560. doi:10.1109/TMI.2018.2867350
5. Komura D, et al. Universal encoding of pan-cancer histology by deep texture representations. Cell Reports 2022;38(9):110424. doi:10.1016/j.celrep.2022.110424
6. Tolkach Y, et al. Artificial intelligence for tumour tissue detection and histological regression grading in oesophageal adenocarcinomas: a retrospective algorithm development and validation study. The Lancet Digital Health 2023;5(5):e265–e275.
7. Johnson WE, Li C, Rabinovic A. Adjusting batch effects in microarray expression data using empirical Bayes methods. Biostatistics 2007;8(1):118–127.
8. Parrish R. Which fixed H&E measurements survive a change of laboratory. Technical note, 2026. https://www.parallaxmetrology.com/Field_Transport_NOTE.html
9. Parrish R. Two pre-registered transport tests of fixed structural H&E measurements. Technical note, 2026. https://www.parallaxmetrology.com/Transport_Tests_NOTE.html