News

Z-sight joins robot action generation with latent world modeling

Maven Robotics and USC researchers introduced Z-sight, a robot foundation model that couples action generation to future visual tokens predicted in its VLM's native feature space.

D
Nov 19, 2026 · 2 min read

Researchers affiliated with Maven Robotics and the University of Southern California introduced Z-sight, a robot foundation model that predicts future visual tokens in its vision-language model’s native feature space as it generates actions.

In the version-one preprint, the authors report an 81.9% average success rate on the SIMPLE humanoid benchmark and 73.3% across six physical Unitree G1 tasks. The findings come from the authors’ own evaluations and have not been independently replicated or peer reviewed in the sources examined.

Z-sight adds explicit foresight without reconstructing future video pixels or using a separate frozen encoder as its prediction target. Its 3.18-billion-parameter mixture-of-transformers combines a 2.22-billion-parameter Qwen3.5-2B vision-language backbone with separate 480-million-parameter foresight and action experts. The foresight expert predicts the visual tokens that the model’s evolving vision encoder would produce for a future observation. The action expert can attend to that prediction as it forms.

At inference, the two experts start with Gaussian noise, then update the future tokens and a 30-step action chunk together over 10 shared Euler steps. The model does not decode the tokens back into pixels. For training, an exponential-moving-average copy of the vision encoder supplies stop-gradient future targets, while the future-prediction loss is blocked from updating the vision-language backbone. The researchers say this setup keeps perception and prediction aligned while limiting representation collapse and the loss of pretrained knowledge.

The authors describe a three-stage training recipe. Pre-training uses 1,977 hours of egocentric human video. Mid-training uses about 2,750 hours of cross-embodiment robot data with human-video replay, bringing the pre- and mid-training source pool to 4,731 hours. Post-training then uses demonstrations from the target robot.

On SIMPLE, the reported 81.9% result spans 12 teleoperated tasks at three difficulty levels, putting it 9.6 percentage points above GR00T N1.7’s 72.3%. The paper says each method was evaluated in three passes, with 10 episodes for every task and difficulty level in each pass.

In the physical G1 evaluation, Z-sight completed 44 of 60 trials, or 73.3%, compared with 53.3% for Fast-WAM. Each method received 10 trials on each of six tasks from matched starting configurations. The paper does not report confidence intervals for those real-world rates.

The researchers also measured a median 109.5 milliseconds to generate a 30-step action chunk on a single RTX 4090, excluding image capture, network transport and low-level control. GR00T N1.7 was faster in that test at 60.0 milliseconds; three other systems in the comparison were slower. The paper also identifies a planning limitation: Z-sight predicts the future from observations alone rather than conditioning foresight on candidate actions, so it cannot yet compare possible action outcomes at test time.

More news