Fixed, named H&E measurements on the external split of the Kather colorectal benchmark: a development record from 85.7% to 94.3% on held-back tiles

Parallax Pathology · Technical note

Russell Parrish · analysis carried out with Claude (Anthropic) · Version 2, 6 November 2026 (version 1, 5 November 2026, reported the first test and the first adjustment)

Abstract. Our earlier tissue-classification figure, 91.83% nine-class accuracy, came from cross-validation inside a 1,800-tile subset of CRC-VAL-HE-7K. Trained instead on the separate NCT-CRC-HE-100K collection and tested on CRC-VAL-HE-7K, the split the dataset authors designed, the same fixed measurements with a linear discriminant readout scored 83.7% on all 7,180 test tiles. The 5,380 test tiles never used in any development were then sealed. Development continued on the training collection and the 1,800 previously used tiles only, and each round ended with one logged evaluation of a frozen, hashed version on the sealed tiles. Over five rounds, sealed accuracy rose from 85.7% to 94.3% (95% tile-level interval 93.6–94.9%) and balanced accuracy from 76.7% to 92.1%. The gains came from four sources: a label-free protection for violet-rendered tiles; extension measurements built earlier for the stroma and muscle boundary; new fixed measurements of tissue colour, texture and nuclear form, chiefly the haematoxylin-to-eosin ratio of non-nuclear tissue; and a label-free protection for orange-rendered tiles. The measurements are fixed and each is a named quantity of the tile. The readout that combines them (gradient boosting averaged with logistic regression) is fitted to the training collection. A round that changed only the readout gained nothing on sealed tiles. Stroma remains the limit (recall 0.65): the test collection's dense, aligned stroma looks closer to the training collection's muscle than to its stroma. The dataset authors' convolutional network reported 94.3% on all 7,180 test tiles; the comparison is approximate, since the sealed subset differs in size and class mix. The sealed tiles have now been evaluated six times, so these figures are a development record; independent confirmation needs a new collection.

1. Why this test

Within-dataset cross-validation with folds split by tile lets tiles from the same patient and slide fall on both sides of a split, so it measures how separable the classes are in one collection, not accuracy on new patients. Kather and colleagues released NCT-CRC-HE-100K as a training set and CRC-VAL-HE-7K as a separate test set from different patients [1, 2]. Fitting on the first and testing on the second is the comparison those datasets were built for, and the comparison published models report.

2. Data, measurements and the evaluation rule

Data. NCT-CRC-HE-100K: 100,000 tiles of 224 × 224 pixels at 0.5 µm/px from 86 slides, nine classes. CRC-VAL-HE-7K: 7,180 tiles from patients with no overlap with the training set (50 patients per the dataset record [2]; the paper describes 25 patients and slides [1]). Both are distributed already Macenko-normalised by their authors. Both archives matched the published checksums.

Development tiles and sealed tiles. 1,800 CRC-VAL tiles (200 per class) had been used in our earlier work, including the error analysis that produced the extension measurements in round 2. After the first external test, the other 5,380 tiles were sealed: no labels, predictions, features or images were opened during development. Every choice was made on the 100,000 training tiles and the 1,800 development tiles. At the end of each round the version was frozen and hashed, then run once on the sealed tiles, and the result was entered in a look log. Looks 0 to 5 are all the evaluations made on those tiles.

What is fixed and what is fitted. Every measurement is a fixed, label-free calculation on one tile: a stain amount, a texture energy, a nucleus width, a gap fraction. The same calculation is applied to every collection, and nothing in it is learned. The readout that maps the measurements to nine classes is fitted on the training collection: a linear discriminant in looks 0 and 1, then gradient boosting (scikit-learn HistGradientBoostingClassifier), from look 3 averaged with logistic regression. Median imputation and standardisation are fitted on the training tiles.

How one tile is read: colour gates, optional re-colouring, measurement, fitted readout
Figure 1. How one tile is read in the final version (look 5). The colour gates and the measurements are fixed and label-free; the readouts (green) are fitted on the training collection. Violet tiles are re-coloured and then read like any other tile; orange tiles go to a separate readout that uses only fields unaffected by re-colouring.

Uncertainty. Wilson 95% intervals at tile level. Patient identifiers are not available for the tiles, so the intervals understate the true uncertainty. Changes between looks are compared tile by tile (paired sign test on the tiles that changed outcome).

3. The first external result (version 1 of this note)

The 81 fixed fields behind the 91.83% figure, with a linear discriminant fitted once on all 100,000 training tiles, scored 83.7% on all 7,180 test tiles (balanced 77.0%) and 85.7% on the 5,380 later-sealed tiles (balanced 76.7%). On the 1,800 development tiles it scored 77.8%, against 91.8% from within-dataset cross-validation on those same tiles. The exact 58-column instrument as published scored the same (83.8%). Fat, background, mucus and tumour transferred well (recall 0.90–0.97); stroma (0.38) and debris (0.45) did not.

4. Five rounds

LookChange (frozen before the look)Accuracy, 5,380 sealed tilesBalancedAgainst previous look: newly right / newly wrong
081 fixed fields, linear discriminant85.7% (84.7–86.6)76.7%·
1+ violet protection87.0% (86.0–87.8)82.0%95 / 28
2+ 73 extension fields; gradient boosting readout91.5% (90.7–92.2)89.4%442 / 198
3Readout only: orientation-free fields, boosting averaged with logistic regression91.6% (90.9–92.3)89.9%113 / 105 (p = 0.64)
4+ 70 new fields (colour, texture, nuclear form); six field views averaged93.8% (93.1–94.4)91.0%234 / 120
5+ orange protection94.3% (93.6–94.9)92.1%29 / 0

All p-values for the paired comparisons are below 10−8 except look 3. Every figure in this table was recomputed from the saved per-tile predictions of each look.

Sealed accuracy and balanced accuracy at looks 0 to 5
Figure 2. Accuracy and balanced accuracy on the 5,380 sealed tiles at each logged look. The dotted line marks the dataset authors' network on all 7,180 test tiles [1] (an approximate comparison).
Recall by class at each look
Figure 3. Share of each class classified correctly at each look (5,380 sealed tiles: adipose 1,138, background 647, debris 139, lymphocytes 434, mucus 835, smooth muscle 392, normal mucosa 541, stroma 221, tumour 1,033).

4.1 Round 1: violet protection

No global stain difference between the collections was found. The debris failure had a specific cause: half of the development debris tiles are rendered in a single violet hue, against under 3% of training debris, and the instrument read them as tumour. Every tile of every collection is checked with one colour rule (mean blue minus mean red above 15). Flagged tiles keep their structure but are re-coloured, each pixel's brightness rank mapped to the average training tissue colour at that brightness, and are then re-measured. Debris recall rose from 0.47 to 0.96; debris called tumour fell from 58 to 1. Corrected Macenko re-normalisation, per-collection feature rescaling and per-channel colour mapping were tried on development tiles and rejected.

Violet debris tiles before and after re-colouring, with normally rendered debris
Figure 4. Development tiles only. Top: violet-rendered debris. Middle: the same tiles re-coloured. Bottom: normally rendered debris.

4.2 Round 2: extension measurements

Two families of fixed measurements (33 and 40 fields), built in August 2026 from an error analysis of the 1,800 development tiles, were added unchanged: multiscale fascicle geometry, coupling between nuclei and fibres, gland-ring polarity, and mixture architecture. They are weak alone (66.7–75.4% on development tiles) and strong in combination with the original 81. With a gradient-boosting readout, stroma recall rose from 0.35 to 0.71 and every class reached 0.71 or above. With the linear discriminant, the same measurements scored 88.0%, so the readout contributed about 3.5 points of the 4.5-point gain.

4.3 Round 3: readout only

Round 3 tuned the readout and dealt with eight fields that change when a tile is rotated (two pairs of direction offsets were replaced by their lengths; the other four were dropped). Averaging gradient boosting with logistic regression gained 1.9 points on development tiles, selected from about 18 combinations. On sealed tiles it gained nothing: 91.6% against 91.5%, 113 tiles newly right and 105 newly wrong. Errors moved between classes (normal called tumour fell from 61 to 48; stroma and muscle confusions rose from 94 to 118). On those 148 measurements the readout had reached its operating limit, and further gains needed new measurements.

4.4 Round 4: colour, texture and nuclear form

A family of 70 new fixed fields measured the non-nuclear tissue directly: absolute stain amounts by fixed colour deconvolution (Ruifrok and Johnston [3]) with no per-tile rescaling, CIELab colour, rotation-invariant texture (local binary patterns, Gabor energy at 4, 8 and 16 px, eosin co-occurrence), nuclear width, length, bluntness and waviness, clear zones at the poles of elongated nuclei, and gaps. Sixty of the 70 fields separate stroma from muscle, or normal mucosa from tumour, in the same direction in both collections. The strongest is the haematoxylin-to-eosin ratio of non-nuclear tissue: smooth-muscle cytoplasm takes eosin strongly, while collagen and ground substance hold relatively more haematoxylin. It separates stroma from muscle in both collections (area under the curve 0.71 in training tiles, 0.86 in development tiles).

Median H/E ratio, non-nuclear tissueMuscleStromaDebrisTumourLymphocytesNormal
Training (NCT)0.390.510.450.630.680.80
Development (CRC-VAL)0.340.530.540.610.570.59

Adipose is omitted: with little tissue per tile, the ratio is unstable (5.4 and 1.08). The ratio also depends on a working stain separation, and reads orange-rendered tissue as nearly unstained (4.5).

A development muscle tile and a development stroma tile split into haematoxylin and eosin, with the non-nuclear tissue that the ratio is measured on
Figure 5. The haematoxylin-to-eosin ratio on two development tiles, each the one closest to the median of the normally rendered tiles of its class. Columns: the tile; the haematoxylin and eosin amounts from fixed colour deconvolution; the non-nuclear tissue the ratio is measured on (pale areas, nuclei with a 2-pixel margin and empty space, are left out). Muscle cytoplasm is dense in eosin; stroma holds relatively more haematoxylin.
Distributions of the H/E ratio for stroma and muscle in both collections
Figure 6. The ratio separates stroma from muscle in both collections. Values above 1.2 are shown at 1.2. Of the 12 development muscle tiles below 0.1, 11 are the orange-rendered tiles of 4.5.

Single readouts with the new fields moved errors rather than reducing them. The gain came from averaging the class probabilities of twelve readouts (gradient boosting and logistic regression on each of six field sets). Three of the six field sets were screened with development labels; a split-half check, screening on one half of the development tiles and scoring on the other, gave 92.1%. On sealed tiles the average scored 93.8%, against 91.6% for a single view on the same fields. Muscle called stroma fell from 70 to 3, normal recall rose from 0.86 to 0.91, and stroma recall fell from 0.72 to 0.64. A new error appeared: 38 sealed muscle tiles called mucus.

4.5 Round 5: orange protection

On development tiles the same error came from muscle rendered orange-tan, a second rendering of the test collection. A label-free gate catches it: the median CIELab b* of tissue pixels is above zero (yellow-shifted), which ordinary pink and purple H&E almost never is; by the corresponding non-nuclear measurement, only 2 of 100,000 training tiles (both background) pass. Re-colouring alone failed: re-coloured orange muscle read as stroma, because the orange rendering has erased the stain balance that separates the two, and no re-colouring can restore it without knowing the class.

The two colour gates plotted for training and development tiles
Figure 7. The two colour gates. Each point is one tile. Violet tiles (purple) lie to the right of the dashed vertical line; orange tiles (tan) lie above the dashed horizontal line. In the training collection the violet gate flags 363 of 100,000 tiles and no tissue tile is orange; among the 1,800 development tiles it flags 103 violet and 23 orange.

Gated tiles are therefore re-coloured, re-measured, and read by a separate readout restricted to the 84 fields that change by at most 0.25 standard deviations when training tiles are forcibly re-coloured: mostly texture, orientation and shape. Every other tile is read exactly as in round 4.

Orange-rendered development tiles, two re-colourings, and normally rendered muscle
Figure 8. Development tiles only. Rows: orange-rendered tiles (debris, then muscle); re-coloured by brightness rank; re-coloured by shifting CIELab a* and b*; normally rendered muscle.

On sealed tiles the gate routed 47 tiles (38 muscle, 5 debris, 4 stroma). None had been correct at look 4; 29 became correct and none became wrong, and the other 5,333 tiles kept their look-4 predictions exactly. Muscle called mucus fell from 38 to 0; 19 muscle tiles are now called stroma, against 3 at look 4.

5. What remains

At look 5, 307 of 5,380 sealed tiles are misclassified, against 769 at look 0. The four largest confusions account for 161 of them.

Misclassified sealed tiles at each look, split by type of confusion
Figure 9. Where the errors went. Each bar is all misclassified sealed tiles at one look, split by the most common confusions. Debris called tumour disappears at look 1 (violet protection); muscle called mucus appears at look 4 and disappears at look 5 (orange protection); stroma called muscle stays.
Confusion matrix at look 5
Figure 10. Look 5 on the 5,380 sealed tiles: dataset label (rows) against prediction (columns); shading by share of the row.
Dataset label → predictionLook 0Look 5
Adipose → muscle (of 1,138)752
Stroma → muscle (of 221)5451
Normal → tumour (of 541)8939
Muscle → stroma (of 392)6319
Debris → tumour (of 139)580
Lymphocytes → debris (of 434)4313

Stroma called muscle has barely moved across the five rounds (54 at look 0, 51 at look 5; lowest 40, at look 2). Development stroma called muscle is dense, wavy and strongly aligned; stroma called correctly is loose and pale; the training collection's stroma is mostly loose, vacuolated or myxoid. Dense fibrous stroma therefore falls on the muscle side of what any readout fitted to the training collection learns. Whether those tiles are a stroma type absent from training, mixed or transitional crops, or labelling differences is a question for a pathologist's blinded read. Adipose called muscle rose from 7 at look 0 to 37 at look 2 and 52 at look 4, and was not investigated.

Stroma and muscle tiles from both collections
Figure 11. Development and training tiles only. Rows: development stroma called muscle; development stroma called correctly; development muscle called correctly; training stroma; training muscle.

6. Reading the result

On tiles it was never tuned on, the instrument went from 85.7% to 94.3%, and from 76.7% to 92.1% balanced. Each gain came from something that can be named: a rendering the training collection lacks, flagged by a colour rule; a stain balance that separates two tissues; texture and nuclear form that survive re-colouring. Each step was followed by a frozen evaluation, and the steps can be read tile by tile. The round that changed only the readout produced no gain. The rounds that added measurements or protections did, though the look-4 gain needed both the new fields and averaging over field views: a single view on the same fields scored 91.6%.

The dataset authors' convolutional network reported 94.3% on all 7,180 test tiles [1]. This instrument reaches 94.3% on the 5,380 sealed tiles. The two figures are not on the same tile set: the 1,800 development tiles are excluded here, and the class mix differs. The comparison is approximate. It does place a fixed, named measurement set with a fitted readout at the level of the learned model on this benchmark.

The two rendering protections are part of the instrument for any collection: both are label-free colour rules, applied to every tile before measurement, and both flag almost nothing in the training collection. They say when a tile's colour should not be trusted, which is useful independently of the classifier.

7. Tried and set aside

8. Limitations

Data and code availability

The lab book for each round is published as written: round 1 (violet protection), round 2 (extension measurements), round 3 (readout only), round 4 (colour, texture and nuclear form), round 5 (orange protection). Frozen specifications, script hashes, per-tile measurements and predictions for every look, and the look log are held by the authors and are available on request. Only public data were used.

References

  1. Kather JN, Krisam J, Charoentong P, et al. Predicting survival from colorectal cancer histology slides using deep learning: a retrospective multicenter study. PLOS Medicine 2019;16(1):e1002730. doi:10.1371/journal.pmed.1002730
  2. Kather JN, Halama N, Marx A. 100,000 histological images of human colorectal cancer and healthy tissue (NCT-CRC-HE-100K and CRC-VAL-HE-7K). Zenodo 2018. doi:10.5281/zenodo.1214456
  3. Ruifrok AC, Johnston DA. Quantification of histochemical staining by color deconvolution. Analytical and Quantitative Cytology and Histology 2001;23(4):291–299.