Depth Holonomy: the instrument

A reader of pictorial space. One picture goes in; what comes out is a set of readings of how that picture organises depth: where its planes are, what it groups, how it holds its field, and where its own local readings refuse to agree. Built 2026-09-21/22.

1. What it is

The reader. A monocular depth model (Depth-Anything-V2-Small, Apache-2.0) trained on millions of photograph-and-depth pairs. Give it one picture and it returns a guess at relative depth for every pixel, using the cues a person uses: what occludes what, how texture thins with distance, contrast falloff, familiar form. Two properties govern everything downstream. It is a guess from a single view, not a measurement: no second camera, no parallax, nothing triangulated. And it is relative: it fixes what is nearer than what, never by how much, because scale and zero point are unknown.

The move. A whole-image reading is one surface and is therefore coherent by construction; it cannot represent a space that contradicts itself, and asked to read one it quietly compromises. So the instrument also reads the picture in small windows, each blind to the rest, and compares. A picture holds only one viewpoint, so displacement of the window stands in for displacement of a camera. Everything else is bookkeeping on those two objects: one global reading, many local ones.

Why it is not a depth tool. Nothing here wants a 3D model. The readouts are about pictorial organisation: planes, groupings, framing, closure, where the reading depends on context. The depth values are a substrate; the structure over them is the instrument.

2. The primitive: the depth reading

3. Readout catalogue

Each readout: what it answers, how it is computed, what it looked like, and where it fails. Earlier readouts (context dependence, holonomy, ordinal order) came from the original question, whether a picture can be locally plausible and globally impossible. Later ones (contours, cliffs, groups, closure) came from what the panels turned out to show.

3.1 Whole-image depth field (the B&W panel)

Answers: how does this picture lay out its space? How: one forward pass, normalised between its 1st and 99th percentiles. Use: the reference reading, and the control everywhere else: a single reading is always coherent, so any incoherence the instrument reports must come from comparing readings. Limit: relative only; smooth; trained on photographs.

3.2 Iso-depth contours

Answers: where are the planes, and how does the picture frame them? How: 14 equal-quantile level sets of the smoothed reading, drawn warm (near) to cool (far). Reads as: framing mechanisms appear as nested closed loops; a tone that carries across a contour is passage. Limit: level count is a choice; thin lines crowd where depth is flat.

3.3 Depth cliff against tone edge

Answers: where did the artist put an edge that is not a spatial edge, and where is there a spatial step with no edge drawn? How: gradient magnitude of the depth reading against the Di Zenzo colour-gradient of the picture, each normalised by its own percentiles, then differenced. Limit, found: in an etching everything is a drawn line, so the blue channel saturates; and the red channel currently fires on smooth ramps (a receding cobbled street scores highest), so it is not yet a passage detector. Needs a step test, not a slope test.

3.4 Regions cut on depth

Answers: what are the masses, independent of colour? How: watershed on the depth gradient, seeded by h-minima, then merged while neighbouring medians differ by less than a tolerance, to about 40 regions. Limit: masses at equal depth merge (a cloth and the table under it); identity would need mass evidence (ISR, Saltimbanques) or a learned segmenter.

3.5 Groups: masses that read as one unit

Answers: what groups with what? How: depth-cut regions that touch and lie within 0.06 of each other in normalised depth are one group. Reads as: in a Cézanne, cloth plus its fruit as one mass, vessels separate, wall as ground; in Piranesi, an arch holding two groups. Use: the shallow-field case, where planes mean grouping rather than recession.

3.6 Field closure

Answers: how does this picture hold its field? Not a classification: a field can be closed on four sides, three, two, one, from the centre, or radially, and the list does not end, so the readout reports how the closure is done and names nothing. How: near = above the 60th percentile of the reading, far = below the 40th; per side, the share of a 3% border strip held by near mass; apertures are far-space runs along the border longer than 8% of a side; tangency is the share of region boundary running inside the strip; truncation is near mass touching the border; radiality is the rank correlation between distance from centre and nearness.

Closure across the whole corpus (132 pictures; descriptive, no test)

Per-image values: out/closure_all.json and out/structure/<key>.json.

3.7 Planes, relief and recession

Answers: how many planes does the layout need, how cleanly do they separate, and how much depth is modelling within a window against recession across the picture? How: exact weighted 1-D k-means (Jenks) over region medians, with K* the smallest K reaching 85% of the variance; relief is the residual spread inside a window after removing its best-fit plane; recession is the 5th-to-95th percentile span of the whole reading. Status: relief ÷ recession does not yet capture "flat plus illusion": Cézanne and the Dutch still lifes both read 0.14. A per-mass version (modelling inside a mass against gaps between masses) is the open candidate.

3.8 Context dependence (the original measure)

Answers: how much does this picture's depth reading depend on how much of it you can see? How: pair each window with one displaced by S/8 and by S/2; ν is the overlap residual no affine gauge explains, in whole-window units; Δν = ν(S/2) − ν(S/8) subtracts the reader's own instability. Result: separates Piranesi's Carceri from his Vedute in two museums (δ = +1.00 and +0.90), but not beyond reader instability under the pre-registered adjustment. Not a localiser: an expert's 63 marked contradictions score at chance (median plate percentile 0.47).

3.9 Holonomy: can the local readings be glued?

Answers: do the local readings agree pairwise up to gauge but disagree around loops? How: on each overlap a symmetric (reduced-major-axis) affine transition, giving an Aff⁺(1)-valued Čech cochain on the nerve of the window cover; a weighted Hodge decomposition splits it into a consistent part, curl on triples, and harmonic (loops bounding no face). Machinery check: cut one reading into the same windows and re-gauge each at random; the cyclic share comes back ≤ 3e-14. Status: secondary throughout; it is the measure the instrument is named after, and the one that did single out Hogarth's Satire on False Perspective (above all 9 controls) where the primary measure did not.

3.10 Ordinal order (legacy)

"A is in front of B" votes from band medians either side of a boundary, then the exact minimum-weight feedback arc set: the least relation-weight whose removal admits a back-to-front layering. Result: reversed (Carceri score lower than Vedute) and not discriminative; kept in the code, out of every hypothesis. The geometric T-junction cue that was to accompany it failed validation (chance agreement with depth on photographs).

4. What is tested and what is not

The full record with methods, freezes and figures is a separate lab book in the project archive. Protocols: PROTOCOL*.md with PROTOCOL*.freeze.json hashes. Results: RESULTS.md and out/RESULTS_*.json.

5. Corpus and rights

86 public-domain images from the Metropolitan Museum Open Access API and 46 from the National Gallery of Art open-data release (CC0 metadata; open-access images), each with source URL, licence flag, retrieval time and sha256 in corpus/records*.json. Images of in-copyright work were never downloaded: corpus/availability_survey.json records where Escher, Picasso, Braque and Gris images exist (Met, Art Institute of Chicago, Cleveland) and that none are public domain. The Art Institute's image server answers scripted requests with a bot challenge, which was not bypassed.

6. Running it

cd depth-holonomy
.venv/bin/python tests/test_algebra.py                 # 8 exact checks of the algebra
.venv/bin/python scripts/depth_gallery.py              # the primitive
.venv/bin/python scripts/continuous_sheets.py [keys]   # contours, cliffs, depth-cut regions
.venv/bin/python scripts/containment_sheets.py [keys]  # groups (and the rejected containment panel)
.venv/bin/python scripts/closure_sheets.py [keys]      # field closure + all of the above
.venv/bin/python scripts/structure_sheets.py           # planes, relief, local vs global
.venv/bin/python scripts/run_split.py dev|test         # the frozen pipeline (refuses if hashes changed)
.venv/bin/python scripts/analyze.py test               # pre-registered analysis
.venv/bin/python scripts/build_labbook.py              # LAB_BOOK.html
.venv/bin/python scripts/build_instrument_doc.py       # this file

The venv uses system site packages (torch) plus transformers. Depth readings are cached by pixel hash in cache/depth, so reruns are exact and cheap: a full structural sheet of 24 pictures takes about a minute. Image keys are met_<objectID> or nga_<objectid>; any key in corpus/records*.json works as an argument.

7. Code map

8. Invariants for anyone extending this

1. Never group by colour. Colour detaches from object across art history. Cut on the depth field, or bring in mass evidence.
2. Keep every quantity gauge-invariant under d → a·d + b, a > 0. Ratios, ranks, percentiles, differences of like things. If a number would change when the reader rescales its output, it is not a measurement of the picture.
3. One reading cannot show incoherence. Any claim about contradiction must compare readings (windows, contexts, readers).
4. Freeze before testing. Protocol, code and corpus hashed; the runners refuse to run on a changed file. Exploratory work is labelled exploratory in the same document as the result.
5. Report the nulls in the same place as the wins. The expert-marking null and the withdrawn instability claim are in the lab book beside the replications; that is the point of the record.
6. Nothing is published. Local files; hosting is opt-in.

9. Adding a readout

A worked recipe, in the order the existing ones were built.

1. Decide what it reads from. The whole-image reading q (normalised, gauge-invariant), the local window charts, or the depth-cut regions. Never colour similarity.
2. Write the quantity in dh/structure.py if it is per-image, taking (rgb, reader) and returning plain floats and arrays. Ratios, ranks, percentiles and differences of like things only, so the number survives d → a·d + b.
3. Add a gauge-invariance test to tests/test_structure.py: build a synthetic field, transform it by two different (a, b) with a > 1 and a < 1, and assert the readout is unchanged. Two versions of the closure bands were rejected this way before one passed.
4. Add a known-answer test where the answer is constructible: a field closed on exactly two sides, a pure ramp with no modelling, a radial field with both signs.
5. Add a panel in a scripts/*_sheets.py file, taking image keys as arguments, and look at 6 to 8 pictures across genres before believing any number.
6. Record the constants in PARAMETERS.md with their role and sensitivity.
7. Write the definition into MATH.md with an evidence label, including the boundary where it fails.
8. If it is to support a claim, write PROTOCOL_<name>.md (primary measure, criterion, kill condition), hash it with the pattern in scripts/freeze.py, and only then run on held-out material.

10. Parameters

Every constant, its role and its sensitivity, is in PARAMETERS.md. The ones most likely to matter:

11. Provenance and reproducibility

Reader: depth-anything/Depth-Anything-V2-Small-hf, Apache-2.0. Larger Depth Anything V2 models are CC-BY-NC, which matters if any of this is ever productised. Determinism: CPU float32 inference was checked to be bitwise repeatable across two passes (max |difference| = 0); MPS differed by 5e-6 and is not used. Readings are cached by sha1 of the exact input pixels plus the model id, so a rerun is exact and cheap.

Environment: Python 3.14, torch 2.12 (dev build), transformers 5.17, scikit-image 0.26, scipy 1.17, numpy 2.4, on macOS. Machinery check: cutting one reading into the window cover and re-gauging each window at random returns a cyclic share ≤ 3e-14 with total energy 13 to 64, so the algebra adds no false signal. Tests: 8 exact checks of the algebra (tests/test_algebra.py), 7 of the structural readouts (tests/test_structure.py), and 20 of relief and per-mass truncation (tests/test_relief.py), including gauge invariance, ramp exactness at the border, ring suppression and an exact-partition check against brute force.

Freezes: six protocols hashed before their runs (Met test, NGA confirmation, Gothic Arch localisation, expert marking, and the two reproducibility ladders); the runners refuse to execute if any hashed file has changed. Data: every image record carries source URL, museum licence flag, retrieval time and sha256.

12. Relief: the reading minus its own ramp

Added 2026-09-23 after the author looked at Degas's Dancers Practicing at the Barre and said the instrument had found neither the dresses, nor the figures, nor the shadows, nor the shapes. He was right, and the reason is measurable. In the reading q, which spans 0.881 between its 5th and 95th percentiles, a dancer's skirt stands only 0.015 to 0.055 clear of the wall beside her, the rail 0.019 to 0.064, a pointe shoe 0.025, while a cast shadow is correctly given nothing (0.009 or less) and the watering can stands 0.213 clear of the floor. The floor ramps 0.384 over the same distance. Everything the artist drew lives inside about 5% of the range, so any cut taken on q returns the ramp.

Relief takes the ramp out: q minus a local linear fit (Gaussian-weighted, degree 1, sigma 0.12 of the short side), iterated with the masses held out until the mask settles. Three simpler trends were rejected by tests/test_relief.py: a blur is biased on a ramp at the border, a white top-hat fails the same way through clamping, and a single-pass fit rings a false mass around every object. The iteration is needed because a mass running off the edge has, near that edge, a surround made of itself, so its relief fades exactly at the frame.

13. Per-mass truncation and riding

Field closure (§3) asked whether near mass touches the border and answered exactly 1.00 on 58% of the 132 Met and NGA images, sd 0.025, because near mass is one connected sheet and it always reaches an edge. On relief masses the question can be asked per mass instead: of a mass's own outline, what share is frame rather than contour. Over 183 images that runs 0.000 to 0.471, median 0.178, sd 0.105, and it is not confounded by how many masses the reader finds (Spearman -0.08) nor by mass area (+0.03). Its companion rides counts contour running alongside the frame without being cut.

14. The ladders: what reproduces, and the baseline that spoiled it

Is a reading a property of the picture, or of the particular sheet of paper? Two pre-registered tests, each frozen before measuring. Ladder 1 (16 Degas: four plates in multiple impressions or states, one composition in two media, four unrelated) measured pair-separation AUC, the probability that two impressions of one plate agree more closely than two unrelated pictures. The relief mask scored 0.97 and no scalar reached the 0.80 bar, exactly as predicted from the earlier NGA finding.

Then the control that protocol should have contained. Two trivial masks were put through the same test: the near half of the reading scored 0.85 and the dark half of the tone 0.86. Both cleared the bar. Most of that 0.97 was the fact that a picture looks like itself.

And then the crops. The 67 images added for this work had never had their pictorial-field crops checked, while the original 132 were checked one by one and needed 15 overrides. Within single ladder families the crops differ by up to 0.150 in aspect ratio, and masks were being compared with only 5% translation slack, so two impressions cropped to different shapes could not register at all. Hand-setting plate marks from thumbnails would have been inventing data (the three Tourny sheets are trimmed differently and two carry a remarque below the image), so each pair is instead registered to the other by a scale search with phase correlation, applied to all three masks identically. That extends the kill to ladder 1 on its own corpus: relief 0.870 against baselines 0.908 and 0.898, margin -0.038, p = 0.44. Four measurements now agree that the relief mask carries no reproducibility beyond a trivial mask, and the single passing number was the first one, taken on unchecked crops with cross-media pairs the baselines find hard.

Two further lessons. Same-plate impressions are the wrong within-condition: etchings from one plate are the same lines on paper, so everything reproduces and the design has no power. And one run was discarded before this one because the analysis code contradicted its own frozen protocol, putting six pairs of different pictures into the "same picture" set; it is kept as out/RESULTS_ladder_defective.json with the correction recorded as an amendment.

15. What this is for, and open questions

1. A second reader. This one now gates the rest. Everything found belongs to the pair (picture, Depth-Anything-Small) until another model (MiDaS, Marigold) reproduces it, and relief is a derived field, so it is more exposed to the model's anatomy rather than less.
2. Relief's content claim. On a ramp-dominated picture it is the only mask that finds the figures. Shown by eye on four pictures, measured on none. The test needs a human reference, a set stratified by R (relief over recession, which identifies ramp-dominated pictures), and the masks shown unlabelled in random order.
3. Separate the ground plane from the figures, which would fix both the truncation conflation and the 30% blob rate with one change.
4. What do the context-dependence maps actually track? They reproduce across impressions but miss expert-marked contradictions. Candidates: hatching-density transitions, reader-unstable zones.
5. Per-mass relief against between-mass gaps, as the real form of "flat plus illusion" for Cézanne.
6. Degas, properly. The stress case was run and the prediction failed: his groups sit at the bottom of the corpus and do not separate from each other (p = 0.23). That comparison cannot be made at all until masses mean the same thing across media.
7. A pixel-level disagreement mask to replace the 17%-of-short-side blocks.
8. Parallax rendering of one plate driven by the global reading and by the local readings: the global holds, the local should tear.