Confidence: plausible proposal. A generative movement system should not treat every sensor update as if the same amount of time has passed. The proposed experiment couples measured elapsed time with a short somatic event memory, then tests whether the system can distinguish a held pause, a delayed response, and the same sequence performed at a different tempo. Existing robot evidence supports time-aware memory; its value for human–AI movement dialogue remains untested.

The provocation

Most interactive pipelines process poses as a sequence of frames. If frames arrive in the same order, the model receives the same history even when network delay, breath, stillness, or acceleration changes how long the sequence occupies. The system sees order but can lose duration.

That omission is not neutral. A hand held near the chest for two seconds may be preparation; held for twenty seconds, it may be rest, refusal, or sustained attention. A partner who answers after 80 milliseconds feels different from one who answers after 800. Treating both histories as equal makes timing an implementation detail when it is part of the movement.

The argument

Physical Time as a Missing Dimension of Robot Policy Memory Dynamics provides a technical opening. Yushen Liang and colleagues argue that common policy memories advance once per observation, implicitly assigning equal duration to each update. Their Temporal Flow Policy instead retains prior events while evolving its memory state according to measured elapsed time. In controlled comparisons with diffusion and flow-matching policies, successful timing required both the intervals and the retained history; supplying time only to a learned gate was insufficient.

The authors report higher average success than recurrent and state-space memories equipped with the same intervals. Memory readouts recover elapsed time since an event and motion information after occlusion, while altering the supplied intervals systematically advances or delays actions. These are results for the paper’s robot tasks and LIBERO evaluations, not evidence about artistic or somatic interaction.

The proposed connection is that duration should become a first-class conditioning signal in body-to-generation systems. This differs from adding timestamps to a log. The model’s internal state must evolve during waiting, not merely receive a number when the next frame arrives.

Current evidence

Three recent releases make the experiment feasible. Temporal Flow Policy offers a method for continuous-time memory. EgoExplorer aligns egocentric video with whole-body motion, finger articulation, touch, and step descriptions across 100 hours of professional activity. EgoWholeView shows that portable multi-camera capture can estimate body and hands in a shared world frame and improve temporal smoothness.

Together they provide components, not a completed system. No cited paper tests whether time-aware memory makes generative feedback feel more responsive, more legible, or more respectful of a participant’s pause. Those are the experiment’s questions.

The proposed experiment

Phase 1: create a duration-contrast movement set

Record 12–20 participants performing short movement phrases under three controlled contrasts: identical order at different tempos; identical pose held for different durations; and call-and-response with varied delays. Capture pose, hand motion, contact when present, and monotonic timestamps from the same clock. Participants annotate what each pause or delay was doing—preparing, listening, resting, resisting, or transitioning—without treating those descriptions as universal labels.

The critical data pair contains nearly identical ordered poses but different elapsed time. This prevents a model from solving the task through sequence order alone.

Phase 2: compare three memories

Train the same conditional generator with three memory designs: a frame-indexed recurrent baseline; a baseline that receives elapsed time as another input; and a continuous-time memory whose state evolves across the measured interval. Keep the generator, training data, and parameter budget as comparable as possible.

Ask each model to produce a modest feedback signal—a low-dimensional visual field, sound-control trajectory, or avatar response—rather than photorealistic video. This keeps the test focused on temporal relation. Introduce dropped frames and irregular capture rates at evaluation time so a model cannot equate frame count with duration.

Phase 3: test temporal meaning, not only reconstruction

Quantitative measures should include onset error, response-delay error, duration-class discrimination, and stability under missing observations. A blinded listening-and-moving study should then ask whether participants can distinguish “answering,” “holding,” and “interrupting” behaviours, and whether the response feels coupled rather than merely fast.

Participants should be able to stop, repeat, and relabel trials. Their descriptions are evidence about the encounter, not ground truth about what a movement means for everyone.

What success looks like

The strongest result would not be a visually impressive demo. It would be a controlled finding that continuous-time memory preserves timing distinctions under irregular sampling better than both baselines, accompanied by participant evidence that the distinction remains perceptible in interaction.

A useful negative result is equally specific: if timestamp input performs as well as evolving memory, the more complex architecture is unnecessary. If objective timing improves but participant judgements do not, the experiment has separated control accuracy from felt coupling. If meanings vary strongly across participants, future systems should preserve personal calibration rather than collapse duration into one universal vocabulary.

This proposal extends the Lab’s explanation of why prediction needs measurable feedback and its guide to provenance-aware body-to-visual pipelines. It also gives the month’s action-relative representation argument a falsifiable temporal test.

The invitation

The field should stop using “real time” only as a speed claim. Publish timestamp sources, sampling irregularities, dropped-frame behaviour, and response-delay distributions. Then ask a more demanding question: does the system remember not only what happened, but how long the encounter has been becoming?

References

  • Liang, Y., Peng, Y., Jin, B., Zhang, T., Zhang, X., Zhou, S., & Chen, Z. (2026). Physical time as a missing dimension of robot policy memory dynamics. arXiv. https://arxiv.org/abs/2610.20524
  • Zhou, Y., et al. (2026). EgoExplorer: A vision-motion-touch observatory for human loco-manipulation in situ. arXiv. https://arxiv.org/abs/2610.18872
  • Hu, Y., He, X., Cui, D., Shi, Y., & Chen, K. (2026). EgoWholeView: Camera-only head-mounted capture of body and bimanual hand motion for embodied demonstrations. arXiv. https://arxiv.org/abs/2610.16256