News

PhyDEX paper puts three VLA models below 3% success

In the PhyDEX paper, three VLA models average less than 3% success on a 35,357-episode throwing benchmark built to test physical reasoning beyond a robot’s reach.

D
Nov 20, 2026 · 2 min read

Researchers led by Hao Vo introduced PhyDEX, a robot-throwing benchmark for physical reasoning in vision-language-action models in an arXiv preprint submitted November 19. Under the benchmark’s full-diversity setting, the paper reports, GR00T N1.7, DynamicVLA and ForceVLA each averaged less than 3% success.

PhyDEX places the destination beyond the robot arm’s kinematic workspace. Instead of carrying an object all the way to its target, the robot must release it and rely on a ballistic flight path. The authors say that makes estimates of mass, center of mass, friction and inertia consequential: those properties affect grasp slip, motion under load, release state and where the object lands.

The design is meant to expose a distinction that contact-based manipulation tests can blur. In tasks such as pick-and-place, insertion or wiping, a gripper can remain in contact with an object through completion. That can make it difficult to tell whether a failure came from a kinematic mistake or a bad estimate of physical properties. A beyond-reach throw forces the policy to act on those properties before contact ends.

The paper says PhyDEX contains 35,357 successful simulated episodes totaling 143.2 hours, recorded at 10 Hz across nine tasks and 100 kitchen scenes. The benchmark pairs uniform-mass, irregular-mass and dynamic-mass objects with single, multiple and moving destination settings. It also randomizes physical parameters independently of appearance: object mass is scaled from 0.5 to 2 times its nominal value, center of mass is offset by as much as 15 millimeters on each axis, and friction is scaled from 0.8 to 1.2.

To generate the data, the authors used a pipeline that proposes grasps, plans collision-free motion and optimizes the throw in a closed loop, retaining an episode only after a simulated success. They then fine-tuned a separate instance of each model for every task and ran 50 evaluation rollouts per policy per task. The paper reports full-diversity average success rates of 0.9% for GR00T N1.7, 0.2% for DynamicVLA and 2.4% for ForceVLA; none succeeded on a moving-destination task. These findings are author-reported, and the preprint does not present an independent replication.

In the paper’s failure analysis, about 89% of failures happened before a throw began. An average of 21.24% of trajectories reached the throwing stage, while overall success was about 1.2%—which the authors calculated as 5.6% of attempted throws. In a separate ablation, giving the models ground-truth physical properties increased the reported success rates by 12.5% for GR00T N1.7 and 24% for ForceVLA, while DynamicVLA was unchanged. The authors present that result as evidence for what they call physical blindness, not as proof that the benchmark isolates physical reasoning from every other model weakness.

PhyDEX is a version-one preprint, and the authors identify several limits. It tests only throwing, randomizes only mass, center of mass and friction under simplified physics, and evaluates three VLA policies. Peer-review status and independent replication were not established from the source reviewed, so the results cannot be generalized to all VLA systems or all forms of physical reasoning.

More news