Two pre-registered transport tests of fixed structural H&E measurements: added information across hospitals in Camelyon17, none detected in a patient-held kidney-grade pilot
Abstract. We asked whether a fixed block of 42 named structural H&E measurements adds information to simple colour summaries when the data used to fit a classifier are separated from the data used to test it. Two tests were specified in writing before any images were downloaded, with fixed features, one fixed linear readout and pre-registered pass rules. In Camelyon17-WILDS (tumour versus non-tumour in 96-pixel patches; 24 slides from 5 hospitals; 11,996 patches), adding the structural block to colour summaries raised slide-averaged AUROC by +0.044, +0.045 and +0.046 when test patches came from separate slide regions, held-out slides and held-out hospitals; all three 99% bootstrap intervals exclude zero. This met one of four pre-registered pass conditions. The other three, which asked for more specific patterns, were not met. In a check run after the verdict, all twenty shuffled versions of the structural columns lowered AUROC in every partition. In a patient-held pilot on CPTAC clear-cell renal carcinoma (low versus high grade; 79 patients; 158 whole slides; 10,112 fields), adding the structural block to colour summaries changed patient AUROC by −0.023 (95% interval −0.085 to +0.031), and adding its non-optical axes to a stronger optical baseline by +0.014 (−0.016 to +0.047). Neither met its rule, and the branch closed as specified. In both studies a plain grayscale-texture family was the strongest single family. In this benchmark the structural measurements are complementary to colour for patch-level tumour detection, including across hospitals; that complementarity did not carry over to weakly supervised patient-level grading.
1. Question and design
A single train/test split can overstate how well image features transfer: patch-level splits can leak slide identity, and holding out whole hospitals lowers performance in Camelyon17 [2]. These two tests ask a narrower question: does adding the fixed structural block to a colour baseline help, and does that help survive as training and test data are pulled further apart?
Both studies share one design. The structural block is the frozen list of 42 axes (15 geometry, 9 relational, 18 optical/kernel) measured after a corrected fixed-target Macenko normalisation [5]. The colour baseline is a fixed set of RGB, optical-density and chromaticity summaries measured on the unnormalised image. Every representation uses the same readout: training-only standardisation followed by shrinkage linear discriminant analysis with equal priors. There is no tuning, feature selection, pretrained encoder or foundation model. The structural measurements are deterministic; the readout is fitted, and every result below is the measurements plus that named readout. Specifications, pass thresholds and close rules were written and hashed before image download, and every later change was recorded as a dated amendment before any predictive result existed.
2. Camelyon17-WILDS: tumour detection across slides and hospitals
2.1 Setup
Data. Camelyon17-WILDS [1, 2]: 96 × 96-pixel patches from lymph-node sections, labelled tumour if the central 32 × 32 region contains tumour. Slides were eligible if they supplied 250 tumour and 250 non-tumour patches, chosen by a fixed hash of the file path. Twenty-four slides qualified, from all five hospitals (7, 4, 6, 3 and 4 slides). Four patches failed normalisation (too little tissue); under a rule fixed before any classifier was fitted, the same four rows were dropped from every representation, leaving 11,996.
Partitions. Four splits of the same cohort, each 60% train, 20% buffer (discarded) and 20% test: P0 random patches; P1 separate regions of each slide, along its main spatial axis, with a buffer band between train and test; P2 held-out slides; P3 held-out hospitals.
Inference. AUROC and balanced accuracy are computed per slide and averaged, so each slide counts once. Intervals come from 20,000 bootstrap draws resampling slides within hospital. Decisions used 99% intervals and a material-effect size of 0.03 AUROC.
Pass rule. Any one of four conditions: (1) performance falls in a graded ladder from random to region to slide splits; (2) adding structure to colour helps by at least 0.03 AUROC in P1 and in P2 or P3, with 99% intervals above zero; (3) structure degrades differently from colour as splits get harder; (4) structure and colour swap rank between easy and hard splits.
Amendments before results. Four, all recorded before any classifier was fitted: an even assignment of slides to folds; decoding the PNG files to RGB as the official WILDS loader does; the complete-case rule above; and fixing the slide-averaged metric as the inferential scale.
2.2 Results
Slide-averaged AUROC
Random patches
Slide regions
Held-out slides
Held-out hospitals
Colour summaries
0.875
0.847
0.858
0.818
Structural block (42 axes)
0.856
0.827
0.849
0.826
Colour + structural block
0.918
0.891
0.903
0.865
Grayscale texture
0.914
0.896
0.907
0.900
Geometry axes only (15)
0.667
0.655
0.676
0.664
Binary edge organisation
0.522
0.511
0.509
0.510
Figure 1. Change in AUROC from adding the structural measurements. Camelyon17 (blue): colour plus structural block minus colour alone, slide-averaged, with 99% slide-bootstrap intervals; the random-patch row (open dot) is descriptive and not part of the pass rule. CPTAC (orange): the two pre-registered patient-level contrasts with 95% intervals. Dotted ticks mark each rule's pre-registered size threshold.
Condition 2 met. Adding the structural block to colour raised AUROC by +0.044 (99% interval +0.004 to +0.092) for separate slide regions, +0.045 (+0.014 to +0.077) for held-out slides and +0.046 (+0.003 to +0.094) for held-out hospitals. Balanced accuracy rose by +0.06 to +0.08 in the same splits. The gain is about the same size under every split, including held-out hospitals.
Conditions 1, 3 and 4 not met. There was no clear ladder from random to region to slide splits (region splits were slightly harder than slide splits), structure and colour did not degrade differently enough to pass, and their ranks did not reverse. These would have been more specific findings; the data do not show them.
Independent recomputation. A separate validation script recomputed all 600 slide- and patch-level cells and the three decisive intervals from the saved predictions, matching the verdict.
2.3 Checks run after the verdict (labelled as such)
Shuffled columns. Twenty controls permuted each of the 42 structural columns independently within hospital, keeping the colour block unchanged. Every one of the 20 controls lowered AUROC in every split (mean change −0.007 to −0.013 by split; the largest control change was −0.003), while the real block raised it by 0.043 to 0.046. This points to the content of the measurements rather than the extra columns. With 20 controls, the smallest attainable empirical p-value is 1/21 = 0.048.
Which part carries it. Adding only the geometry, only the relational or only the optical/kernel axes to colour gave smaller gains (about +0.005 to +0.033) than the full block, all with 99% intervals that include zero. These are descriptive and do not localise the effect to one family.
Question. Camelyon's pass permitted one further test, under its own frozen specification: does the same complementarity hold with real patients held out, on whole slides, for a different task? The specification also asked a stricter version: do the non-optical structural axes (geometry and relational, 24 axes) add anything beyond a strong optical baseline (colour plus grayscale texture plus the 18 optical/kernel structural axes)?
Data. CPTAC-CCRCC [3], read as DICOM whole-slide images from the NCI Imaging Data Commons [4]. Eighty patients with grade G1 to G4 and two tumour slides each were chosen by a fixed hash within grade before any image was downloaded (G1 5, G2 35, G3 31, G4 9), giving 40 low-grade (G1/G2) and 40 high-grade (G3/G4) patients. Up to 64 non-overlapping 512-pixel fields at 0.5 µm/px were taken per slide by a fixed tissue mask and farthest-point order; a slide needed at least 48. One patient was excluded by this rule (one slide gave 43 fields), with no replacement, leaving 79 patients (39 low, 40 high), 158 slides and 10,112 fields. All fields were measured successfully.
Readout and inference. Five patient-held folds, each originally 8 low- and 8 high-grade patients. Fields inherit their patient's grade label; field probabilities are averaged within slide and then across the patient's two slides. Patient-level AUROC with 20,000 class-stratified patient bootstraps; decisions used 95% intervals.
Pass rules. Mechanistic pass: non-optical axes add at least +0.020 AUROC to the strong optical baseline with a 95% lower bound above zero, and adding the full block to colour is not worse than −0.010. Transport-only pass: adding the full block to colour gains at least +0.030 with a 95% lower bound above zero. Otherwise close the branch and do not download the remaining patients, tune features or add baselines.
3.2 Results
Representation (patient level)
AUROC
Balanced accuracy
Colour summaries
0.651
0.607
Structural block (42 axes)
0.577
0.545
Colour + structural block
0.628
0.608
Grayscale texture
0.683
0.633
Strong optical baseline
0.675
0.633
Strong optical + non-optical axes
0.689
0.646
Transport contrast: −0.023 AUROC (95% interval −0.085 to +0.031; positive in 1 of 5 folds).
Mechanistic contrast: +0.014 (−0.016 to +0.047; positive in 3 of 5 folds).
Disposition: scientific close. Neither rule was met. The remaining 71 patients were not downloaded. An independent validation of 37 checks recomputed the metrics, fold structure and verdict.
Post-primary control. Twenty permutations of the 24 non-optical axes within training patients gave increments from −0.011 to +0.013 (mean −0.001); none reached the observed +0.014 (empirical p = 1/21). This was fixed in advance as a sensitivity check that cannot reverse the verdict, and it does not.
4. Reading the two together
The structural block adds information to colour for patch-level tumour detection, and the gain is about the same when whole hospitals are held out. It added no detectable information for patient-level grade in kidney cancer, where every representation was modest (patient AUROC 0.58 to 0.69) and labels were inherited by every field from the patient. The two tests differ in organ, task, field size, label type and aggregation, so this pair maps an operating window rather than isolating why one passed and one did not.
In both studies a plain grayscale-texture family was the strongest single family (Camelyon 0.90 to 0.91; CPTAC 0.68). Any claim for the structural measurements has to clear that baseline, not only colour. On Camelyon, colour plus structure roughly matched grayscale texture except under held-out hospitals, where texture alone was higher (0.900 against 0.865).
5. Limitations
Camelyon17-WILDS tests patch-level tumour detection on 24 slides; its patient field is not treated as an independent patient level. A hospital effect combines scanner, stain, population and annotation differences.
The Camelyon pass establishes complementarity to one colour summary under one linear readout. It does not separate tissue architecture from preparation, and it does not show the structural block beating stronger baselines.
The CPTAC pilot has 79 patients and one grade endpoint; grade can covary with tissue composition, tumour purity, necrosis and specimen handling. Closing this branch is a study decision under its rules, not evidence that every structural representation or pathology endpoint would fail.
Both tests were analysed by the same programme that developed the measurements, with independent recomputation of the results but not independent replication on new data.
Data and code availability
Frozen specifications, amendments, scripts, predictions, validation records and hashes are held by the authors (folders CAMELYON17_WILDS_PARTITION_GATE_V1 and CPTAC_CCRCC_PATIENT_HELD_PILOT_V1; this note in TRANSPORT_NOTE_20261105) and are available on request. Only public data were used. Analysis carried out with AI assistants.
References
Bándi P, et al. From detection of individual metastases to classification of lymph node status at the patient level: the CAMELYON17 challenge. IEEE Transactions on Medical Imaging 2019;38(2):550–560. doi:10.1109/TMI.2018.2867350
Koh PW, et al. WILDS: a benchmark of in-the-wild distribution shifts. Proceedings of the 38th International Conference on Machine Learning, PMLR 139, 2021.
Clark DJ, et al. Integrated proteogenomic characterization of clear cell renal cell carcinoma. Cell 2019;179(4):964–983.
Fedorov A, et al. National Cancer Institute Imaging Data Commons: toward transparency, reproducibility, and scalability in imaging artificial intelligence. RadioGraphics 2023;43(12):e230180. doi:10.1148/rg.230180
Parrish R. A missing sign check in a Macenko stain normaliser collapsed haematoxylin and eosin into one direction and created a site difference in TCGA colorectal measurements. Technical note, 2026. https://www.parallaxmetrology.com/Stain_Normaliser_NOTE.html
