Parallax Metrology

Spatial Kernel · Embodied-Verification Instrument

Does it look right,
or is it right?

A learned success judge answers "does this look correct?" The Spatial Kernel answers "is it correct?" deterministically, from pixels, with no model in the loop. It measures the actual centroid of visual mass against the target an instruction named, then adds an independent two-view depth cross-check.

On real photographs it caught ten flat printed decoys a frontier embodied VLM certified as real 3D objects present. Perception said PASS. Geometry said FLAT.

Primary result 10 flat decoys caught · real 3D vs. flat print separated within a stated envelope Capture 19 object types · ordinary phone · two ~10 cm-offset views · no depth sensor Engine vtl_spatial · deterministic · no learned judge · same input, same output Pre-registration adversarial-symmetric protocol scaled to 1,200
10
Flat decoys caught
Printed objects a frontier embodied VLM certified as real 3D objects present. The two-view geometric check caught every one.
19
Object types
Real mugs, a paper cup, a Scotch-tape box and more, shot on an ordinary phone from two ~10 cm-offset views. No depth sensor.
0
Learned judges in the loop
Every coordinate is standard computer vision. No trained metric, no bespoke operator. Same input, same output.
67.7%
Evidence the judge missed
On a public corpus an object's true visible extent moved 67.7% while its learned box moved 2.4%, the change the judge reasons over never registered.

A spatial instruction
is a geometric claim.

Spatial Kernel Paper — Primary

Displacement, plus an independent depth cross-check

Robots can see and robots can plan. What no standard pipeline instruments is whether the spatial intent of a natural-language instruction became a geometric fact. The Spatial Kernel measures the actual centroid of visual mass and compares it to the target the instruction named, producing a displacement and a compliance verdict without a learned judge. It then adds a second, independent read: a two-view parallax cross-check that separates a real 3D object from a flat printed decoy. Success is treated as the agreement of those independent reads. Across 19 object types photographed on an ordinary phone, the geometric check caught ten flat decoys a frontier embodied VLM certified as real objects present, within a stated envelope reported honestly alongside the cases where it declines.

The Diagnosis + The Forward Study — Context

Why a learned judge cannot hold the metric

The companion diagnostic paper, Confidence Without Correctness, shows why the independent read is necessary. A frontier success judge asked whether a manipulation succeeded returned false passes at a flat 0.95 confidence, right or wrong, because its learned representation smooths over the sub-threshold physical change that decides the outcome: on a public corpus an object's learned box moved 2.4% while its true visible extent moved 67.7%. The forward study, an authority-separated auditor specification, states which parts of the response already stand on evidence and pre-registers the single experiment that would settle the rest. Together they frame the Spatial Kernel as one instrument with a diagnosis behind it and a decisive study ahead of it.

Two-view overlay of a flattened Scotch-tape box printed to read as three-dimensional; the object marker tracks the table plane between the two views, and the instrument reads it FLAT.
Two-view parallax on a flattened Scotch-tape box printed to read as three-dimensional. Between the two ~10 cm-offset views the object marker tracks the table plane, not an object standing above it (relative parallax < 0.8 px; orange = object, blue = table plane). The instrument reads FLAT, the same object a frontier VLM certified as a real 3D object present.
Four panels of a woman reading by a window: the original photo; a gradient mass-and-skeleton map; the same image marked with a measured centroid at delta-x minus 0.143; and a radial safety overlay.
How the displacement read is built: original frame, gradient mass-and-skeleton map, the measured centroid of visual mass (Δx = −0.143), and a radial overlay. The kernel measures where the mass actually is, not where a caption says it should be.

Every verdict is a
set of countable coordinates.

Coordinate Reads Role in the verdict
Δx, Δy
Centroid displacement
Offset of the measured centroid of visual mass from the target the instruction named. The primary compliance axis: how far the placement actually landed from where it was asked to be.
target mass
Mass on target
Fraction of visual mass that actually falls inside the target region. Separates "some of it got there" from "the placement is genuinely on target."
outside leakage
Mass off target
Fraction of mass sitting outside the target region. The complement that a symmetry- or cluster-fooled read would otherwise hide.
rₖ
Void ratio
How much empty space surrounds the placed mass. Distinguishes a compact placement from one dispersed across the frame.
parallax
Two-view depth
Relative displacement of the object marker against the ground plane across two offset views. Near-zero parallax means the object is flat: a printed decoy, not a real 3D object.
verdict
PASS / FAIL / FLAT
The composite compliance decision with its score. Reported with the envelope in which it is valid, and an honest decline where the evidence is insufficient.

Four Modes Of Spatial Non-Compliance

Infographic titled Four Modes of Spatial Non-Compliance with four labeled panels: Cluster Bias, Peripheral Dispersion, Symmetry Without Occupancy, and Partial Placement Override. Each shows object mass, a measured centroid, a target location, and a correction vector; each is labeled Perception PASS, Geometry FAIL.

Real pixels.
A frontier VLM caught.

Real Captures · 19 Object Types · Ordinary Phone
Ten flat decoys a frontier embodied VLM certified as real objects present.

Real objects photographed from two ~10 cm-offset positions with no depth sensor, the second view is the geometry. Across 19 object types the deterministic two-view parallax check separated real 3D objects from flat printed decoys within a stated envelope. A frontier embodied VLM, run on the same scenes, certified ten flat decoys as real objects present. The kernel read every one of them FLAT.

Flat decoys caught10
Object types19
Depth sensornone — two-view parallax
Learned judgenone
From Confounded Demo To Conformant Instrument
It was measuring the scene, not the object.

The package shipped as a clean CLI that claimed to answer "did the robot put the object where the task required." Reading the source showed it did not: it scored the whole frame, not the target object. Isolating the object before the kernel runs, without adding a learned judge, turned a silently-confounded metric into a conformant instrument, fixed honestly and adversarially against a real frontier VLM.

Original metricwhole-scene, confounded
Fixisolate object, no learned judge
Protocoladversarial-symmetric, pre-registered
Confidence Without Correctness · Public Corpus
A learned judge, confidently wrong and unmoved by being wrong.

Asked to verify whether a manipulation succeeded, a frontier success judge returned false passes at a flat 0.95 confidence, right or wrong. The cause is representational: on Language-Table an object's true visible extent moved 67.7% while its learned box moved only 2.4%. The judge reasons over the box, so the deciding change is invisible to it. An independent pixel measurement recovers exactly the evidence the learned judge discards, and abstains when blind.

Judge confidence (right or wrong)0.95 flat
True visible extent moved67.7%
Learned box moved2.4%
Four Modes · Perception PASS, Geometry FAIL
Four ways a scene reads compliant while the geometry says otherwise.

Cluster bias, peripheral dispersion, symmetry without occupancy, and partial placement override: four distinct failure modes where a scene reads compliant to perception while the measured centroid of mass says it is not. Each is a case where "looks right" and "is right" come apart, and each is caught by measuring the actual mass against the named target rather than trusting the perceptual impression.

Failure modes characterized4
Common signaturePerception PASS, Geometry FAIL
Discriminatormeasured centroid vs. target
A top-down bin-sorting scene captioned 'Place all tools in the bottom-left bin.' A vision system marks the tools as being in the lower-left area, while the kernel marks the measured centroid still near frame center and reports a compliance gap and correction vector.
"Place all tools in the bottom-left bin." Perception agrees the tools read lower-left. The kernel measures the actual centroid of mass: it sits near frame center, well short of the target, and the instrument reports the compliance gap and a correction vector. Looks right; is not right.
Six frames of the P18 episode showing the robot arm moving and a yellow block being progressively uncovered, with learned detection boxes overlaid.
The diagnosis, episode P18: as the arm moves and a yellow block is progressively uncovered, its true visible extent changes dramatically while the learned detection box barely moves. The success judge reasons over the box, so it does not see the change that decides the outcome.

Source frames: Language-Table dataset, C. Lynch, A. Wahid, J. Tompson, T. Ding, J. Betker, R. Baruch, T. Armstrong, P. Florence, "Interactive Language: Talking to Robots in Real Time," arXiv:2210.06407 (2022). Detection-box and actor overlays added by the diagnostic paper.
The claim boundary, stated directly.

The Spatial Kernel is a deterministic compliance instrument, not a general 3D reconstruction system or a replacement for perception. The two-view parallax check separates real 3D objects from flat print within a stated envelope: it requires two adequately offset views and adequate texture, and it declines rather than guesses where those conditions fail. The ten-decoy result comes from documented real captures, not a statistical sample from a known population; the four non-compliance modes are characterized categories, not a proof of completeness. The favorable partial-visibility rule that would have raised one boundary case from 1/12 to 10/12 is rejected, not adopted, and any such change becomes a new prospective question rather than a post-hoc repair. The central claim, robustness scales with the number of genuinely independent observables, is part true by construction and part empirical, and the forward study is designed to pin down which is which.

stated
envelope
valid where two
offset views and
texture exist
abstains declines rather
than guesses when
blind
no learned
metric
standard vision
primitives, same
input same output

Run it beside
your judge.

A learned success judge answers
Does this look like it succeeded?

A learned judge produces a verdict and a confidence from a representation that has already smoothed away sub-threshold physical change. It is fast and general, and it is the right tool for semantic questions. The problem is that it will return a confident PASS on a scene where the deciding evidence never entered its representation, and its confidence does not move when it is wrong.

The Spatial Kernel answers
Did the spatial intent become a geometric fact?

The kernel does not judge intent. It measures the centroid of visual mass against the named target and cross-checks depth across two views, deterministically, from pixels. Low disagreement with the judge means the verdict can be trusted. High disagreement is the warning: "looks right" and "is right" have separated, and this episode deserves a human or a second surface before it enters training.

The intended relationship is diagnostic and complementary. Run the Spatial Kernel beside the learned judge that labels your rollouts. Where they agree, proceed. Where they disagree, you have found a candidate poisoned label, the exact episode a data engine should not train on. The natural place a partner comes in is the third surface: telemetry or a second viewpoint for the strata where a single camera is structurally out of envelope, which is the question the pre-registered forward study is built to answer on real data rather than assert.