KineMask gives video models an initial push, then predicts collisions

Mohamed bin Zayed University of Artificial Intelligence researchers introduced KineMask, a method that conditions a pretrained video diffusion model on an object’s initial direction and speed. From that starting point, the model generates the object’s later path and any resulting collisions instead of following a route specified by the user.

That starting condition is what sets KineMask apart from drag-based video controls. Systems such as TORA require a complete drag trajectory, while MotionI2V is also evaluated with a trajectory. KineMask receives only an initial object velocity, leaving the later path, destination and interaction outcome unspecified.

The control is encoded as a three-channel “velocity mask” over the selected object’s pixels, representing motion along the x, y and z axes. At inference, a user draws an arrow that the system converts into the mask. A ControlNet branch passes the signal to an image-to-video diffusion backbone, which remains frozen during ControlNet training.

Training takes place in two stages. First, the model receives velocity masks throughout each training video. The second stage fine-tunes the control branch while randomly removing later masks, teaching it to operate when only the first frame’s velocity is available. The researchers generated two Blender datasets for the work, Simple Motion and Interactions, each with 10,000 training videos and 100 test samples.

KineMask also uses a second conditioning signal. Training videos are paired with descriptions of their dynamics, and at inference GPT-5 produces a likely scene-outcome description from the input image and requested motion. The paper says this text improves visual realism and object consistency. In an ablation where text and motion cues conflicted, the researchers said the velocity signal still affected whether an interaction occurred.

On the researchers’ synthetic Interactions benchmark, unmodified CogVideoX recorded a Fréchet Video Distance of 807.3 and a Fréchet Video Motion Distance of 3,514.9. KineMask trained on the Interactions data recorded 250.7 and 143.8, respectively, where lower is better. The KineMask team reported these results; the sources reviewed for this article did not include an independent replication.

The authors also ran pairwise comparisons with 30 participants on a 50-image real-world set. They reported that participants preferred KineMask over the tested baselines for motion fidelity, interaction realism and overall physical consistency. Because the set has no ground-truth motion trajectories, the comparisons measure participant judgments rather than objective physical correctness.

The team says changes produced by different starting velocities and by replacing a struck object show that the model captures collision causality and responds to apparent mass. The experiments do not establish that KineMask learned an internal Newtonian physics system. They show that outputs changed in ways the authors considered consistent with altered starting conditions.

The primary implementation uses CogVideoX-I2V-5B, with further tests on Wan2.2-5B and Cosmos2.5-2B. The research project page identifies the work as an ECCV 2026 paper. For related context, see MBZUAI research on parallel decoding for video object tracking.

KineMask’s low-level control is limited to velocity; it does not explicitly control friction, mass, shape or air resistance. Training used static cameras and focused on rigid-body center-of-mass motion rather than part-level actuation. The authors also document failures in which short objects are ignored instead of producing a collision, while crowded scenes with similar objects can cause duplication or disappearance.