NereidVLA brings language-guided control to underwater robot navigation
Researchers introduced NereidVLA, a vision-language-action model that converts visual histories and language instructions into local 3D trajectories for closed-loop underwater navigation.
Researchers from Zhejiang University and four partner institutions introduced NereidVLA, a vision-language-action model for underwater navigation. It converts four recent RGB frames and a language instruction into a local three-dimensional trajectory. A low-level controller follows part of the predicted path before the model replans from new camera observations, closing the loop between perception and movement.
The approach separates marine spatial learning from robot-motion training. The researchers first fully fine-tuned Qwen3-VL-Instruct backbones into a model called Sea2Know using about 2 million marine vision-language examples drawn from existing datasets and simulated scenes, without paired robot trajectories. They then initialized NereidVLA from the 8-billion-parameter version of Sea2Know and adapted it with expert trajectories executed under simulated underwater-vehicle dynamics.
NereidVLA represents motion as 10 discrete tokens selected from a fixed 1,024-entry displacement codebook. Those tokens decode into 10 body-relative waypoints spanning five seconds. The model receives only visual history and language; robot state estimates support training targets and feedback control but are not policy inputs.
For navigation training, the team says it built 573 simulated MarineGym scenes and collected 5,117 expert demonstrations across obstacle avoidance, target search, pipeline navigation and cave navigation. It extracted 100,000 training samples from 4,597 demonstrations in 516 scenes. The paper also introduces a six-part underwater-navigation understanding benchmark containing 3,646 questions retained after three annotators reviewed 4,500 candidates.
On that author-created benchmark, the researchers report that Sea2Know-8B beat its Qwen3-VL-8B starting model in all six categories. The gains included 21.83 percentage points for spatial relations, 28.97 points for temporal understanding and 15.26 points for local action decisions. In offline trajectory tests on held-out simulated scenes, NereidVLA recorded the lowest error on four of five reported metrics, although its 95th-percentile average displacement error was slightly worse than SimpleVLN’s.
In 500 closed-loop simulation trials per model, the authors report an 84.4% overall success rate for NereidVLA, compared with 72.4% for Qwen3-VL-8B under matched trajectory representation, supervision and execution settings. InternVLA-N1 slightly outperformed it on pipeline navigation alone, at 87.2% versus 86.4%.
The researchers also tested a BlueROVHeavy in a laboratory pool without fine-tuning on that environment. NereidVLA completed 63 of 80 trials across four tasks, a reported 78.75% success rate, versus 47.50% for InternVLA-N1 and 35.00% for Qwen3-VL-8B. The paper is an arXiv preprint, and the benchmark, simulations and pool trials have not been independently replicated; the study does not establish performance in open water, on longer missions or on other robot platforms.
More news

MuJoCo 4.0.1 corrects solver bug from major release

Z-sight joins robot action generation with latent world modeling

RECLAIM uses robot attention feedback to counter adversarial vision patches
