Two AI models can use latent spaces with the same size, architecture, and statistical shape yet assign different coordinates to the same movement. A “shared latent space” is therefore not automatically a shared language. Direct exchange requires corresponding actions to land in corresponding places—and that must be measured, not inferred from the models’ design.

A latent space is an internal map, not a universal atlas

Imagine that two dancers each draw a private map of the same studio. Both maps use a square page and the same scale. One places the entrance at the lower left; the other rotates the room and places it at the top. Each can navigate perfectly using their own map. Handing one dancer the other map’s coordinates, however, will send them in the wrong direction.

A latent space works similarly. It is a compressed coordinate system learned by an artificial-intelligence model. In a motion variational autoencoder, or motion VAE, an encoder turns a movement clip into latent coordinates and a decoder turns those coordinates back into movement. The model is rewarded when its own encoder and decoder cooperate. Nothing in that objective necessarily requires another model’s decoder to interpret those coordinates in the same way.

“Shared” can mean three different things

The 12 October preprint What Makes a Latent Space Shared? separates three ideas that are often bundled together.

First, recoverable correspondence means a transformation can be learned between two coordinate systems. The maps differ, but a translator can connect them. Second, direct compatibility means corresponding movements already occupy comparable coordinates; no translator is needed. Third, decoder reuse means a code produced under one schema can be passed through another schema’s decoder and still reconstruct the intended motion.

These are progressively stronger requirements. If a linear calibration aligns two spaces, correspondence was recoverable. That does not prove the raw codes were directly interchangeable, and direct retrieval does not prove that crossed decoding will produce good movement.

The experiment exposes the difference

Iago Brito and colleagues tested a multi-schema motion VAE on HumanML3D. Even when identical motion inputs were encoded through the compared schemas, raw nearest-neighbour retrieval—Recall@1—was only 5.9% to 17.7%. Paired-token cosine similarity ranged from 0.221 to 0.484. Yet a learned linear calibration recovered the correspondence almost perfectly.

In plain language: the same movements were present in both maps, but the coordinates were arranged differently. A simple translator could expose the match, while direct exchange failed.

Cycle consistency did not solve the problem. Training a round trip—go from one representation to another and back—improved recoverability, but direct retrieval and token agreement remained near zero. By contrast, only 64 paired clips raised raw Recall@1 to 88.4%–90.3% and token similarity to about 0.82 in the reported setting. Small amounts of explicit correspondence supervision reorganised the maps much more effectively than architectural symmetry alone.

The authors also found that crossed decoding remained substantially worse than native reconstruction. Even better coordinate agreement did not make every decoder freely reusable.

A concrete movement example

Suppose one schema describes a person with 22 skeletal joints and another describes a robot with different limb proportions and constraints. Both encoders may compress “step left, turn, and reach” into a 256-dimensional vector. Matching vector length tells us only that both maps have 256 axes. It does not tell us that axis 17 means turning energy in both systems, or that a point decoded for the human will be physically executable by the robot.

This matters in creative tools too. A performer may want a movement captured by one system to drive an avatar, a sound model, and a generative visual model. If each system claims to use a shared latent space, the interface can still drift: a sharp pause may become smooth, a small weight shift may disappear, or a culturally meaningful quality may be reduced to a generic style code.

Somatic-AI Lab’s October frontier report describes this as an action-relative evaluation problem: a representation should be judged by the distinction the next task needs. Its earlier synthesis on structured motion representations asks the companion question: what disappears when movement is divided into particular units?

How should “shared” be evaluated?

Ask four separate questions. Can a lightweight calibration recover paired examples? Do corresponding movements retrieve each other without calibration? Can one model decode the other model’s codes? And does the exchanged result preserve the movement quality relevant to the person or task?

The last question cannot be answered by coordinate similarity alone. A high cosine score may coexist with a visible timing error; a lower score may preserve the contact or direction that matters. Evaluation therefore needs both representation-level tests and task-level consequences.

The paper studies a controlled VAE setting, so its numbers should not be treated as universal thresholds. Its broader lesson is robust and practical: common dimensions, shared weights, or matching distributions describe how systems were built. Compatibility describes what happens when information actually crosses the boundary.

The takeaway

A shared latent space is a hypothesis about communication, not proof of it. Before connecting motion systems, identify whether the goal is translation, direct retrieval, or decoder reuse; provide paired evidence appropriate to that goal; and test what the resulting movement preserves. Two maps can describe the same world without using the same coordinates.

References

  • Brito, I., Rios, W., Fernandes, L., de Oliveira, K., & Galvão Filho, A. (2026). What makes a latent space shared? Direct and recoverable compatibility in motion VAEs. arXiv. https://arxiv.org/abs/2610.17376
  • Guo, C., et al. (2022). Generating diverse and natural 3D human motions from text. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. https://doi.org/10.1109/CVPR52688.2022.00509