Gaussian and colour controls for PathoROB and CRoMa

Russell Parrish · analysis carried out with AI assistants

What should a robustness score look like for features drawn without reading the image? How different is it when the only image information is colour? We provide runnable references for both questions on the published TCGA, Camelyon and Tolkach evaluation designs. Six RGB statistics scored below all three Gaussian seeds on every cohort under RI, MaRI and CRoMa. The lower tail shows where a headline or a count of nonpositive scores can lose important information.

GitHub release v3.0.0 · Download the code, archived scores and run instructions · Extended methods and results

What the experiment measures

We compared benchmark-labelled tiles from different slides. One type of neighbour matches the query’s biological class across centres. The other matches its centre across biological classes. RI counts these two types within a fixed neighbourhood. MaRI weights the evidence by feature distance. CRoMa compares the mean cosine distances to the five closest eligible neighbours of each type. A negative margin means the same-centre type was closer on average. A zero margin is a tie. Neither constitutes a diagnostic error. The original authors introduced the metrics; our contribution is a reproducible control comparison. PathoROB · CRoMa.

We regenerated 42-dimensional Gaussian vectors (three fixed seeds) and measured the mean and population standard deviation of each RGB channel on 99,392 public source tiles. The scoring manifests use 88,650 distinct tiles. TCGA contributed 112,800 occurrences from 59,250 tiles in 94 paired 2×2 comparisons. Camelyon contributed 20,400 tiles from two centres, and reduced Tolkach ESCA 9,000 tiles from three. We used the original within-cohort scaling for colour, slide exclusion, neighbourhood sizes (61, 11, 46 respectively) and automatic MaRI temperature. Labels denote cancer project on TCGA, normal or tumour on Camelyon, and annotated tissue class on Tolkach.

A design check counted eligible same-class/cross-centre and different-class/same-centre candidates for every query. For uniformly sampled eligible neighbours, the ratio of the expected informative counts is 0.500 on TCGA, 0.500 on Camelyon and 0.3014 on Tolkach. The corresponding three-seed RI means are 0.5002, 0.4992 and 0.3003. This ratio explains why a universal 0.5 reference would misread Tolkach. It approximates the expected ratio; it is not an exact expectation for a finite random graph.

Reading the lower tail

Camelyon separates coverage from severity. RGB reached tissue-class balanced accuracy of 0.794 while its RI was 0.009. Only 12,935 of 20,400 tiles had an informative fixed-k neighbour. Among the other 7,465, CRoMa assigned 6,971 nonpositive margins. Yet 2,033 of the 2,040 most negative margins were on tiles that already had fixed-k support. The distance margin both recovered missing coverage and distinguished severity among covered tiles.

Tolkach separates frequency from severity. Gaussian had more nonpositive margins than RGB (84.6% versus 73.6%). The mean among the lowest tenth was −0.086 for Gaussian and −0.922 for RGB. The count tells how often the margin falls at or below zero. The tail tells how far it falls.

TCGA illustrates the sampling unit. The RGB headline was −0.168. Its lower-tail mean of −0.868 came from 11,280 occurrences representing 8,878 distinct tiles across all 94 pairings. Each tile could appear once per pairing. A tail occurrence is not an independent patient.

Claim and scope

These controls calibrate three representations of neighbour evidence on fixed benchmark designs. They do not partition the causes of a centre difference among staining, acquisition, case mix or annotation. The comparison used six colour features and a 42-dimensional Gaussian family, so it cannot isolate the contribution of dimension alone. Three seeds describe simulation variation. No inference on diagnostic accuracy, treatment, survival or patient harm was tested.

Reproducibility

The example pins source revisions and image-shard hashes and checks both the native and comparison outputs. The 10th-percentile tail includes all ties at the cutoff. On Camelyon many local RGB RI scores are exactly zero, so the tie-inclusive RI tail can contain far more than a tenth of defined tiles. Each implementation is verified against its own archived run. Native and comparison RI differed by up to 0.000009 at nearly tied neighbour cutoffs. The release contains scripts, numerical references and input specifications. Obtain the public tiles under their source terms. Runnable artifact.

Archive SHA-256 61e9fbe8f655175609a377a929adcba1c96f3a18c9cafa1409441dd415eff4a6