Ai2 releases ToyBlocks as a controlled test world for robot learning

Ai2 released ToyBlocks, an open simulated tabletop environment built for controlled robot-learning experiments and the first system in a broader framework it calls FoundationWorlds. A robot arm can push, flip, rotate, pick up, place and stack blocks. The bounded domain is designed to let researchers vary training data or model design and trace the source of failures.

Ai2 says its first dataset contains 85,336 long-horizon demonstrations representing roughly 1,600 hours of robot execution and 144 million frames. Those figures come from Ai2’s release materials and have not been independently replicated.

ToyBlocks is deliberately narrower than open-ended robotics settings. Ai2 says the smaller range of objects and interactions makes it practical to combine realistic physics and appearance, automatically generated tasks and demonstrations, reasoning-and-dexterity challenges, and broad coverage of possible tasks within the domain.

Researchers can change one variable—such as the mix of pushing, flipping or pick-and-place examples—while holding other parts of an experiment fixed. That includes evaluating models on tasks or workspace regions excluded from training.

The data-generation pipeline begins with task goals expressed in Planning Domain Definition Language, or PDDL, a formal way to describe a starting state, available actions and a goal. It samples scenes, uses task-and-motion planning to select and sequence robot skills, executes plans across parallel simulated environments, checks each step, renders the results and writes datasets in the LeRobot format.

The framework separates high-level reasoning from low-level motion control. A high-level model chooses the next skill and target block; a low-level policy drives the arm.

During evaluation, either layer can be replaced with an oracle, a component that supplies the correct decision. This allows researchers to test whether a long-horizon failure came from planning or physical execution. Beyond task success, ToyBlocks measures execution speed, forces on objects and partial task progress.

Ai2 reported that atomic ManiFlow skills reached 80.0% success in simulation and 81.1% on a physical robot. In a hierarchical test combining Molmo 2 with ManiFlow, the system completed 111 of 200 simulated episodes and 15 of 30 physical-robot episodes.

Ai2 cautioned that more trials are needed. The 30 physical episodes used blocks seen during training and the easiest scene configuration, so the release does not establish broader real-world generalization.

Ai2 said fully integrated reinforcement-learning training is not part of the initial package and is planned for a later release. The core ToyBlocks repository is public under the Apache License 2.0, although submodules and downloaded assets can carry separate licenses.