Last updated: 2026-07-24
TL;DR: MolmoMotion predicts future 3D point trajectories from a short visual history, query points on an object, and a language instruction. Ai2 released model variants, weights, code, training data, and a benchmark for research. The approach is relevant to robot planning and controlled video generation, but the published results come from the project team and the model still uses a sparse set of points with limited closed-loop evaluation.
What MolmoMotion predicts
A motion tracker explains where an object moved. MolmoMotion addresses the next step: where selected points on an object are likely to move after an instruction. Its output is expressed in a metric 3D world frame, so the same path is intended to remain meaningful across changes in camera view.
The project describes two variants. MolmoMotion-AR emits quantized coordinates in sequence, while MolmoMotion-FM uses flow matching to generate continuous trajectories. Both use Molmo 2 as the vision-language backbone that connects the instruction to the target object and its query points.
Released research assets
Ai2’s June 17 release article links the model collection, MolmoMotion-1M data, PointMotionBench, the technical report, and public code. MolmoMotion-1M was assembled from 1.16 million videos and pairs action descriptions with object-grounded 3D trajectories. PointMotionBench contains 2,700 human-validated clips spanning 111 object categories and 61 motion types.
The project team says it built the data pipeline because existing collections did not combine large-scale video, object-grounded 3D paths, and action descriptions. The pipeline tracks points in 2D, lifts them into a shared 3D frame, filters incoherent motion, smooths trajectories, and clips each sample around the meaningful action.
Robotics and video use cases
For robotics, a predicted object path can initialize or guide a manipulation policy. Ai2 reports that a policy initialized from MolmoMotion reached a higher pick-and-place success rate than its Molmo 2 baseline in the team’s MolmoSpaces experiment. That is a source-reported research result, not a guarantee for a different robot, camera, object set, or controller.
For video generation, predicted trajectories can act as an explicit motion-control signal. The paper evaluates this by conditioning an image-to-video system on the paths and comparing several motion metrics. Teams should reproduce the result on their own scene types before treating it as evidence of general video quality.
Readers comparing research releases through AI News should keep the distinction between an open research artifact and a supported production service. MolmoMotion does not arrive with a hosted commercial SLA, a managed API price, or a promise that the benchmark covers a buyer’s environment.
Limits that matter
The project page says MolmoMotion uses eight query points per object during training. That sparse representation cannot describe dense surface geometry, and complex deformable motion remains difficult. The authors also identify broader real-world and closed-loop robot evaluation as future work.
Teams should inspect the Apache 2.0 licensing details for the released artifacts, the data documentation, input preprocessing, compute requirements, failure cases, and any safety boundary around a physical controller. A forecast should be treated as a model output that a control system can reject, not as a command that automatically reaches hardware.
Research artifact versus product
The public release is valuable because its paper, code, weights, data, and benchmark can be inspected together. It still requires engineering work to reproduce the environment, prepare inputs, choose query points, measure latency, and connect a forecast to a downstream system. There is no published managed-service commitment covering uptime, support, privacy, or production incidents.
That distinction should shape the evaluation. A research team may accept manual setup to study a new representation. A product team needs an owner for model updates, data provenance, monitoring, rollback, and the operational cost of keeping the pipeline working.
Practical evaluation
- Reproduce one published example with the released checkpoint and code.
- Inspect how query points and camera calibration affect the predicted path.
- Test rigid, articulated, and deformable objects separately.
- Compare both model variants on accuracy, latency, and compute.
- Keep any robot trial behind collision checks, speed limits, and human supervision.
Official sources
Related Kingy AI links
Kingy Launch Brief
Put the week’s verified AI launches in your inbox.
Get a source-checked briefing on consequential AI launches, with a clear try, watch or skip verdict. Beehiiv will ask you to confirm your address, then you can choose the subjects you want to follow.
Free · Choose your subjects · Double opt-in · Unsubscribe anytime