News

A2-VLA tests unfamiliar tools before steering robot policy

A2-VLA tests proposed tool functions in reconstructed simulations, then turns validated spatial features into guidance for a vision-language-action policy.

D
Nov 20, 2026 · 2 min read

Chenyuan Zhou, Chaoyi Huang, Zibo Wang, Zhiyuan Li and Yong-Lu Li introduced A2-VLA, a framework that tests how unfamiliar tools can perform a task before using the results to guide a vision-language-action policy. The authors report improved success with unseen tools across ring-moving, sand-cutting and plate-lifting tasks. The work is an arXiv preprint, and the paper does not report an independent reproduction.

A2-VLA begins with teleoperated demonstrations using known tools. It reconstructs the tools, target objects and spatial arrangement in simulation, then treats a tool’s task-relevant affordance — the part and orientation that make it useful — as a hypothesis. The agent proposes functional regions and geometric properties, builds extraction skills and retargets demonstration trajectories to candidate tools. Simulated task outcomes, contact information and diagnostic views help it revise those proposals.

When an extraction skill succeeds, the system saves it along with the conditions under which it applies. That skill can then annotate training demonstrations and extract corresponding features from unseen tools. For each task, A2-VLA represents the affordance with 12 coordinates: the functional region’s position, a sign-invariant orientation descriptor and a task-specific direction. A learned encoder turns the vector into a token added to the image, language and robot-state context for the pi-0.5 policy backbone. An action expert uses that context to generate action chunks.

Training combines an action objective with an auxiliary objective that predicts how the affordance will move. The authors also used 10% counterfactual retargeting data, intended to teach the policy to respond when an affordance pose changes. At deployment, the agent reconstructs an unseen tool, tests and refines its affordance parameters in simulation, and updates the spatial features during execution as the robot observes and replans.

The paper reports simulation retargeting success of 81.3% for Pnp Ring, 93.8% for Cut Sand and 83.3% for Reset Plate, for an unweighted mean of 86.1%. The strongest displayed comparison mean was 37.8% for a raw vision-language model. The evaluation used 80 episodes each for Pnp Ring and Cut Sand and 60 for Reset Plate. Failed mappings and unreachable plans counted as failures.

Real-robot tests used a Dobot Nova 2 bimanual platform and three RealSense D405 cameras. On unseen tools, the authors report mean success of 35.3% for the full system, against 4.7% for pi-0.5 alone, 25.0% for agent-only control and 22.0% for a version without simulation iteration. A2-VLA achieved 46% on Pnp Ring, 26% on Cut Sand and 34% on Reset Plate. Each task and tool set used 50 episodes for VLA-based methods and 20 for agent-only control.

The authors limit the approach to tool-transfer problems that trajectory retargeting can handle. They also say the VLA policy can fail even with correct affordance guidance. The paper reports no confidence intervals or statistical-significance tests for the real-robot comparisons.

More news