Your robot doesn't know where the table is. Neither does its training data — we checked.

Here's a question that sounds trivial until you try to answer it from a public robotics dataset: when the robot puts a fork on the table, how far is the gripper from the surface?

Classical robotics answers this with forward kinematics, hand-eye calibration, and a depth sensor — everything lands in one coordinate frame and the distance is a subtraction. But many training exports do not carry that answer in a portable form. They preserve pixels and joint states while omitting the calibration and world geometry needed to interpret them metrically — and recovering it from RGB alone is mathematically ambiguous (a small table up close and a big table far away can produce identical pixels).

The research community has noticed. SpatialVLA bolts a depth model onto the vision tower. PointVLA injects point clouds into a frozen action expert. A 2026 result showed that feeding a single grounded 3D point directly to the action head unlocks spatial generalization that architecture tweaks couldn't. These results show that explicit 3D signals can improve generalization. They also raise a data question: how often do training datasets preserve those signals?

We measured it.

What we ran

We audited seven representative LeRobot-format datasets on Hugging Face — 9,694 episodes combined — with traceplane check v0.5.9 plus a scan of each dataset's declared observation streams in meta/info.json. Three questions per dataset:

1. Does it declare camera calibration (intrinsics/extrinsics) — the thing that relates pixels to the robot's frame?
2. Does it declare a depth-like stream? (We separately assess whether its measurement source and metric scale are declared — that's question 3's territory.)
3. Does it carry a versioned spatial artifact — queryable metric relationships between the robot and the world, with provenance?

To be precise about what this is and isn't: calibration and depth results are declaration-level — we scanned each dataset's meta/info.json. We separately probed the two Traceplane artifact paths (meta/spatial_context.json + spatial/relations.parquet — our own format; LeRobot does not currently define an equivalent artifact), but did not inspect other sidecars or dataset cards. LeRobot can store additional features, but it currently defines no conventional schema for calibration, derived spatial relationships, uncertainty, or provenance. Those signals were absent from this sample.

The results

Across seven representative LeRobot datasets totaling 9,694 episodes: none declared calibration fields in meta/info.json; six declared no depth stream; none contained a versioned spatial artifact.

Three details make this more interesting than a simple "nobody bothered":

1. The geometry existed upstream and didn't survive conversion. DROID — the original — collects calibrated stereo from three cameras with per-scene extrinsics. It's one of the most geometry-rich real-robot datasets ever released. The LeRobot port (droid_100) declares three left-eye RGB streams: no stereo pair, no depth, no calibration fields. The conversion to the training-friendly format is where the declared spatial context disappears.

2. Even the one dataset declaring depth can't ground it. USC-GVL/humanoid-everyday declares observation.depth.egocentric — to their credit, the only depth stream in the set. But with no intrinsics and no camera-to-robot extrinsics declared alongside it, using only the declared dataset fields, that depth map cannot be projected into the robot's frame — you may know a pixel is 0.4m from the lens, but not where it is relative to the gripper. (External calibration, a URDF, or undocumented mounting information could make the projection possible, but none is declared in the dataset; nor did we validate the depth stream's contents — whether it is sensor-measured and metrically scaled is exactly the kind of provenance question these formats can't currently answer.)

3. The sim datasets had perfect ground truth for free. pusht and the ALOHA sim tasks were generated inside simulators that knew every object pose to machine precision. The exported datasets declare pixels and joint states; the poses aren't among the declared features. Simulation is the one place where metric spatial context costs nothing — but with no standard schema telling exporters where to put poses, they didn't keep them.

(We also pointed the tool at RealSourceData/RealSource-World, NVIDIA's GR00T-X sim collection, and apple/ml-egodex; their nested/custom layouts don't resolve as single LeRobot datasets, so they're excluded from the counts rather than counted against.)

Why this matters for training

A policy that decelerates before contact must represent proximity somehow. Without metric geometry, it may rely on appearance and motion cues tied to a particular camera setup, weakening transfer across viewpoints and embodiments. This is the well-documented weakness of 2D-only VLAs on precise manipulation, and it's why the 3D-injection papers above report their gains on spatial generalization benchmarks specifically.

The common retrofit — running a monocular depth model like Depth Anything over the RGB — deserves precision: the default relative-depth checkpoints produce scale-ambiguous predictions, and while metric variants exist, their outputs remain learned estimates rather than sensor measurements. Both are useful; they are not interchangeable; and today's datasets lack a standardized field to record which one you have.

What "having it" looks like

We built spatial context as a first-class dataset artifact in traceplane. One thing to be clear about up front: Traceplane does not reconstruct geometry that was never captured. It standardizes, materializes, and validates calibrated 3D signals already produced by your capture or perception pipeline — tracked 3D points, planes, calibration, transforms. If your rig measured it, we make it a queryable, certified artifact. If it was not measured, a perception model can estimate it — and Traceplane preserves that distinction as estimated provenance rather than letting it pass as a measurement.

pip install traceplane

# Declare your frames, calibration, and the relationships you care about,
# then materialize the relation table:
traceplane spatial ./my-dataset --config spatial.yaml

# Validate geometry, provenance, and uncertainty:
traceplane check ./my-dataset --spatial

That writes two artifacts next to your data:

• meta/spatial_context.json — an SE(3) frame graph (camera ↔ robot base ↔ world), calibration, and the declared metric-scale provenance: observed (sensor-measured), estimated (model-guessed), or simulation (ground truth).
• spatial/relations.parquet — one row per episode/timestep/relation: gripper-to-object distance, signed distance to the table plane, approach speed, time-to-contact, confidence, uncertainty. Queryable with SQL like any other table.

And because a distance derived from RGB-D + calibration is not interchangeable with one guessed by a monocular model, provenance is now part of the certification badge at cert.traceplane.ai: spatial: measured, sim, estimated — or no pill at all, which is what every dataset above would show today.

The ask

If you publish robot datasets: keep the calibration. It's kilobytes. If your capture rig measures depth, ship it — and declare it. If you're exporting from sim, export the poses you already have. The models are getting better at using 3D; the bottleneck is that the data keeps arriving without it.

If you want to know where your dataset stands: traceplane check <path> --spatial is one command, and this audit is reproducible from the script and pinned revisions below.

Methodology: traceplane check 0.5.9 (PyPI; install verified in a clean environment on publication day), metadata-level validation (check_data=False), anonymous Hugging Face access, 2026-07-23. "Declared depth" and "Declared calibration" come from each dataset's meta/info.json feature list ("Declared calibration" = any feature name matching intrinsic/extrinsic/calibration/transform patterns); the two Traceplane artifact paths were checked separately as existence probes. Episode counts as declared by each dataset's metadata. HF main branches are mutable, so results are pinned to the revision SHAs recorded at audit time — the audit script and the frozen result set with per-dataset SHAs are published alongside this post. Note: USC-GVL/humanoid-everyday has since been renamed to USC-PSI-Lab/humanoid-everyday; its artifact absence was verified directly against both artifact paths after its check-phase run hit anonymous rate limits.